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CHAPTER  1 


DIGITAL  SPEECH  COMMUNICATIONS  TEST  AND  EVALUATION  OVERVIEW 


l.I  LISTENER  TEST  AND  EVALUATION  ^  DIAGNOSTIC  RHYME  TEST  (DRT) 


The  Department  of  Defense  has  selected  the  DRT  (Refs.  1  -  3)  as  the 
standard  test  for  intelligibility  of  digital  voice  communication 
systems.  This  test  method  has  been  used  by  the  DoD  Digital  Voice 
Processing  Consortium  for  the  past  ten  years  as  the  primary  measure  of 
intelligibility  of  secure  digital  communication  systems.  RADC/EEV, 
NRL,  NSA,  DCA  and  many  others  have  also  used  the  DRT  as  an  aid  to 
their  individual  efforts  in  speech  compression  research.  NSA  has 
selected  the  DRT  as  the  final  measure  of  intelligibility  for  the 
Future  Secure  Voice  Program  and  as  a  selection  procedure  for  the  STU 
ITI  contractor.  All  of  this  interest  stresses  the  importance  of  the 
DRT. 

Independent  DRT  measures  are  available  from  Dynastat  Corp.  of  Austin, 
TX.  Several  years  ago  the  AF  saw  the  need  for  a  government  controlled 
implementation  of  the  DRT  to  serve  as  a  second  source  of  testing  and 
as  a  means  of  verifying  the  test's  repeatability  and  accuracy.  This 
RADC/EEV  implementation  of  the  DRT  is  detailed  in  Reference  4. 

ARCON  Corporation  has  been  heavily  involved  in  the  in-house  DRT 
program  at  the  RADC/EEV  Speech  Processing  Facility  since  its 
inception.  ARCON  has  been  responsible  for  conducting  and  administering 
the  DRT  sessions  for  eight  years.  The  scoring,  reporting  and  analysis 
software  for  the  DRT  results  have  all  been  developed  by  ARCON 
personnel.  Several  new  analysis  tools  have  been  designed  and 
implemented  by  ARCON.  The  ability  to  measure  the  intelligibility  of 
communication  systems  corrupted  by  random  burst  errors  that  totally 
disrupt  a  transmission  channel  has  been  provided  by  ARCON  to  RADC/EEV 
through  the  development  of  a  modified  DRT  method  and  scoring 
procedure.  The  introduction  and  validation  of  a  method  to  equalize  the 
presentation  level  of  DRT  input  tapes  to  voice  processors  based  on 
Equivalent  Peak  Levels  is  an  example  of  ARCON 's  expertise  in  this 
field.  The  modernization  of  DRT  data  collection  with  the  introduction 
of  a  TRS-80  MlOO  Data  Entry  System  was  conceived,  designed, 
implemented,  tested  and  verified  by  ARCON  personnel.  The  development 
of  software  for  these  efforts  can  be  followed  through  ARCON  reports, 
References  5-7. 
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This  report  will  cover  improvements  to  the  RADC/EEV  Test  and 
Evaluation  System  and  several  special  projects  that  were  completed 
during  the  period  of  contract  F19628-86-C-t)057  (15-February- 1986  to 
24- June- 1988) .  Numerous  software  packages  are  discussed  within  this 
report.  Source  code  for  most  of  these  programs  is  available  directly 
from  RADC/EEV.  Any  requests  should  be  addressed  to  the  technical 
contract  monitor,  Mr.  Luigi  Spagnuolo.  A  copy  of  this  report  and  all 
source,  taskbuild  and  overlay  files  required  to  generate  tasks  for 
these  programs  are  available  on  the  RADC/EEV  PDP-11/44  in  virtual  disk 
DRTDEV. 


1.2  RADC/EEV  DRT  SYSTEM  DATA  BASES  AND  SOFTWARE 

This  report  will  discuss  all  modifications  and  additions  made  to  the 
DRT  System  by  ARCON  during  the  current  contract  period.  Chapter  Two 
will  discuss  the  DRT  Data  Base  structures  and  give  details  on  the 
latest  version  of  the  DRT  Analysis  Data  Base  (DB) .  The  BASIC  program 
DRT  that  runs  on  the  Tandy  MlOO  Data  Entry  Units  for  presentation  of 
the  DRT  to  the  Listener  Crew  along  with  the  programs  COM  and  MODEL  for 
transfer  of  raw  DRT  listener  responses  to  the  PDP-11/44  have  not  been 
changed  during  this  contract  period  and  will  not  be  discussed.  Details 
of  these  programs  can  be  found  in  References  6  and  7 .  Chapter  Three 
will  discuss  modifications  to  the  Fortran  program  DRT  that  scores  the 
DRT  data  and  adds  DRT  systems  to  the  comparative  analysis  DB.  Numerous 
modifications  and  additions  have  been  made  to  the  DRT  Comparative 
Analysis  Package  MENU.  Chapter  Four  will  provide  full  details  on  these 
changes.  Chapter  Five  will  complete  the  discussion  on  software  changes 
with  details  on  the  DB  utility  packages.  This  chapter  will  introduce 
the  new  program  GENDB  that  allows  for  the  generation  of  special  probe 
data  base  volumes. 


1.3  SPECIAL  PROJECTS 

Interrupted  Voice  Communication  Systems  -  The  use  of  the  DRT  for  the 
measurement  of  random  channel  effects  has  a  limitation  associated  with 
periodic  presentation  of  the  information  being  tested  (i.e.  the 
leading  consonant  of  the  rhyming  word) .  This  limitation  could  be 
severe  for  tests  measuring  burst  bit  error  and  packet  loss-effects. 
Intelligibility  of  continuous  messages  could  be  very  different  from 
that  measured  by  the  DRT.  A  modification  of  the  DRT  method  has  been 
developed,  implemented  and  tested  by  ARCON  for  the  evaluation  of  these 
"interrupted"  communication  systems.  Software  modifications  to  the 
RADC/EEV  DRT  System  that  allow  for  the  testing  of  interrupted  systems 
are  discussed  in  Chapters  Two  through  Five.  The  development  of  the 
interrupted  system  testing  methodology  and  the  validation  of  this 
method  are  covered  separately  in  Chapter  Six  of  this  report. 
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In-Field  Diagnostic  Rhyme  Testing  -  ARGON  originally  proposed  the  idea 
of  an  In-Field  DRT  capability  in  a  March-84  memo  to  RADC/EEV.  It 
detailed  some  ideas  for  an  In-Place  DRT  Procedure  using  the  TRS80  MlOO 
Data  Entry  Units.  The  test  method  was  suggested  for  use  with  the  ESD 
ANDVT  evaluation  to  be  held  that  summer.  The  idea  was  well  received  by 
ESD  and  RADC/EEV  management,  and  the  project  was  officially  started  at 
an  April-84  meeting  with  ESD. 

Requirements  for  the  full  In-Field  DRT  System  were  supplied  by  ARGON 
including  software  and  analog  recording  equipment.  Procedures  were 
designed  for  the  use  of  in-house  ESD  and  RADC  personnel  as  listeners 
and  speakers.  Training  procedures  were  developed  and  carried  out  with 
these  individuals.  An  in-field  set  of  training  procedures  were 
developed  for  use  with  operational  personnel.  These  procedures  were 
augmented  with  an  audio  ANDVT  DRT  Training  Tape  developed  by  ARGON. 
All  the  required  software  was  developed  by  ARGON,  the  test  equipment 
was  acquired,  base  line  data  was  collected  in-house  and  actual  field 
data  was  collected  by  the  RADC  and  ESD  team.  Validation  measures  on 
this  effort  proved  inconclusive  due  to  numerous  problems  with  the 
field  collection  of  data. 

Documentation  of  the  method,  equipment,  training  procedures,  and 
software  developed  for  the  In-Field  DRT  is  provided  for  the  first  time 
in  Chapter  Seven  of  this  report. 


DRT /DAM  Digital  Mastering  Project  -  ARGON  has  been  involved  in  a  joint 
RADC/EEV  and  DoD  Digital  Speech  Processing  Consortium  effort  to 
digitally  master  the  DRT/DAM  test  material  for  the  past  five  years. 
The  procedures  and  equipment  used  in  this  effort  are  documented  in 
Chapter  Eight  of  this  report.  A  complete  listing  of  the  new  digital 
master  liurary  is  provided  as  an  appendix  to  this  report. 
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CHAPTER  2 


RADC/EEV  DIAGNOSTIC  RHYME  TEST  DATA  BASE  STRUCTURE 


When  a  DRT  session  is  administered,  the  raw  input  data  is  transn.itted 
from  the  Radio  Shack  TRS-80  Model-lOO  Portable  Computers,  which  are 
used  by  a  DRT  listener  crew  for  the  entry  of  a  DRT  booklet  set,  to  a 
raw  data  file  located  on  the  PDP-11/44  DRT  account  at  RADC/EEV.  This 
raw  data  file  is  given  a  file  name  in  the  form  NNNN . ZFY ; W ,  where  NNNN 
will  be  the  previously  assigned  system  number  and  W  will  be  a  version 
that  distinguishes  more  than  one  testing  of  the  same  system  audio 
tape.  All  these  NNNN. ZFY  files  make  up  the  DRT  Raw  Data  Base  (DB)  and 
reside  either  on  the  DRT  account  on  the  PDP-11/44,  on  a  backup  device 
BK: ,  or  on  archived  RL02  disks.  The  structure  of  these  files  has  not 
been  modified  and  is  fully  detailed  in  Reference  7. 

The  routine  DRT  takes  data  stored  in  the  DRT  Raw  data  base,  generates 
scores  (i.e.,  percentage  of  words  correctly  recognized),  processes 
these  scores  into  reports,  and  if  requested,  stores  the  processed  data 
on  another  DB  known  as  the  DRT  Analysis  Data  Base.  The  addition  of  a 
processed  DRT  system  to  the  DRT  Analysis  DB  allows  for  comparison  of 
that  system  to  any  other  in  the  DB.  The  data  bases  are  found  on 
virtual  disk  volumes  stored  primarily  on  the  CDC  300  Mbyte  disk  drive 
at  the  RADC/EEV  Speech  Facility  and  can  be  accessed  by  mounting  the 
virtual  disk  assigned  to  the  DB  requested. 

A  detailed  description  of  the  DRT  analysis  data  base  structure  along 
with  two  special  modifications  to  this  structure  are  presented  in  the 
following  sections. 


2.1  ^  ANALYSIS  DATA  BASE  STRUCTURE 

The  "Data  Base  Disks"  are  setup  to  contain  a  series  of  100  systems 
(i.e.,  2400  to  2499  on  2400  DB  disk)  and  up  to  200  total  repeat  tests 
of  systems.  A  DB  disk  can  be  set  up  using  the  program  DBTILTY  which 
will  be  described  in  a  later  section. 
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Each  DB  disk  contains  three  unformatted  direct  access  files  that  allow 
individual  records  to  be  read.  The  files  are  DB.INX,  DB.HDR,  DB.DAT  as 
defined  in  the  assign  and  define  file  statements  below. 

CALL  ASSIGN  (!.'  DB.DB.INX  ') 

CALL  ASSIGN  (2,'  DB;DB.HDR  ') 

CALL  ASSIGN  (3,'  DB:DB.DAT  ') 

DEFINE  FILE  1  (  3,  200,  U,  lAV) 

DEFINE  FILE  2  (  300,  100,  U,  lAV) 

DEFINE  FILE  3  (  21600,  20,  U,  lAV) 

DB.INX  provides  a  record  of  all  systems  present  on  the  DB  and  any 

repeat  systems  by  transferring  this  record  information  into  three 
arrays,  IDDISK,  INDEX,  and  lOVFLO.  The  IDDISK  array  holds  the  two 
system  numbers  that  define  the  range  of  the  DB  and  is  primarily  used 
as  a  data  base  flag.  In  other  words,  the  routine  can  tell  whether  the 
DB  assigned  to  DK2 :  or  DK3 ;  is  an  original  probe  disk,  an  expanded 
probe  disk,  a  generic  probe  disk,  or  a  system  data  disk  simply  by 
checking  the  values  stored  in  IDDISK.  The  INDEX  array  holds  the  first 
version  numbers  of  the  100  systems  and  the  total  number  of  higher 
versions  that  each  system  has  on  that  DB.  The  lOVFLO  array  holds  the 
higher  version  system  number  for  up  to  200  repeat  tests. 

DB.HDR  contains  header  records  for  each  of  the  maximum  300  systems  on 
the  DB.  The  DB.DAT  file  has  21600  records  of  40  bytes,  each  record 
being  directly  readable.  Each  record  contains  the  36  bytes  that  define 
the  results  of  a  DRT  Booklet  (i.e.,  listener*speaker  interaction  ). 
The  36  bytes  represent  six  features*three  subfeatures*two  state.3(P/A) 
=  36.  Note  that  although  the  main  feature  result  is  a  function  of  the 

subfeature  results,  it  is  given  separately.  This  keeps  the  data 
compatible  with  previous  work. 

The  21600  records  of  DB.HDR  represent  12  listeners*6  speakers*300 
systems  =  21600.  The  position  of  the  particular  system  number  and  its 
version  in  INDEX  or  lOVFLO  acts  as  a  pointer  to  the  location  in  DB.DAT 
from  which  the  record  for  that  system  should  be  read  from.  Read  and 
write  procedures  and  data  formats  can  be  clarified  by  checking  the 
code  in  subroutine  ADDDB  of  program  DRT  and  the  program  package  MENU. 


2.1.1  Interrupted  DRT  System  Data  Base  Structure 

The  development  of  a  modified  DRT  procedure  for  the  intelligibility 
testing  of  "interrupted"  communication  systems  is  covered  in  Chapter 
6.  It  was  decided  that  these  systems  should  reside  in  the  existing  DRT 
analysis  DB.  This  would  allow  for  the  comparative  analysis  of  these 
systems  with  all  other  DRT  systems  in  the  DB.  These  systems  require 
the  storage  of  additional  information.  For  "interrupted"  systems  all 
test  words  are  not  scored,  therefore  ._he  number  of  measures  for  each 
feature  is  not  always  the  same.  The  actual  feature  test  item  counts 
must  be  saved  in  the  DB.  Since  interrupted  DRT  systems  are  added  to 
DRT  DB  and  scored  in  a  similar  manner  as  regular  DRT's,  a  need  for 
distinguishing  between  the  two  types  is  also  necessary. 
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ARGON  solved  the  problem  of  distinguishing  between  systems  with  the 
introduction  of  the  interrupted  system  flag  stored  in  the  unused 
portion  of  each  system's  header.  We  also  employed  the  unused  portion 
of  the  header  to  store  the  test  item  count  values  so  they  could  be 
retrieved  for  analysis  purposes. 

The  unused  elements  of  the  IHEAD  array  (from  the  system's  DB.HDR  file) 
are  used  to  record  the  interrupted  system  flag  and  the  test  item  count 
values.  IHEAD  (52)  is  a  flag  to  indicate  whether  the  system  is 
interrupted  (-1)  or  non- interrupted  (not  -1).  Thus  compatibility  with 
the  previous  non- interrupted  systems  should  be  maintained.  Each  count 
value  can  range  from  0  to  8  as  only  the  four  subfeatures  by  P/A 
categories  are  recorded  for  each  feature.  (Note  the  number  of  test 
items  for  a  "main"  feature  category  is  the  sum  of  two  subfeature 
counts.)  These  values  are  packed  in  4-bit  structures  ordered  first  by 
speaker  and  then  by  feature  just  like  the  actual  DRT  data.  A  total  of 
6*6*4=  144  values  are  recorded  in  36  elements  of  IHEAD  beginning 
at  IHEAD(53). 


2.1.2  Expanded  Probe  Data  Base  Structure 

A  probe  system  is  a  system  run  at  the  start  of  every  DRT  session  and 
is  used  to  monitor  listener  and  soundroom  performance.  An  Expanded 
Probe  Disk  structure  was  created  to  allow  for  more  probes  to  be  stored 
on  an  individual  disk.  Since  the  probe  system  is  the  same  every  time 
it  is  run,  system  numbers  and  higher  versions  have  no  significance  to 
a  Probe  DB.  Probe  disks  are  identified  with  having  system  numbers 
starting  at  9000.  A  new  "expanded"  probe  data  base  was  created  that 
did  not  change  the  structure  of  the  data  base  but  only  changed  the  way 
data  would  be  entered. 

Expanded  probes  have  300  version  1  systems  on  a  DB  where  as  original 
probes  have  100  version  1  systems  with  space  for  200  repeat  tests  of 
the  same  probe  system  (i.e.  higher  versions).  Each  expanded  probe  disk 
is  made  up  of  the  three  unformatted  direct  access  files,  DB.INX, 
DB.HDR,  and  DB.DAT,  as  described  above.  Yet  in  the  expanded  probe  the 
file  DB.INX  is  used  differently  for  the  storage  of  system  numbers. 
System  numbers  on  an  expanded  probe  disk  range  from  9000  to  9299.  The 
two  arrays  in  DB.INX,  INDEX  and  lOVFLO,  are  interpreted  differently 
since  they  are  all  version  1  systems.  The  INDEX  array  now  holds  the 
first  100  probe  systems,  namely  9000  to  9099,  with  the  total  number  of 
higher  versions  of  those  systems  which  should  all  be  set  to  zero  since 
there  are  no  higher  versions  on  an  expanded  probe.  The  lOVFLO  array 
allows  for  200  more  probe  systems,  namely  9100  to  9299.  This  Expanded 
Probe  DB  structure  has  required  changes  to  the  scoring  program  DRT, 
the  analysis  package  MENU,  and  the  DB  utility  routine  DBTILTY. 
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2.1.3  Generic  Probe  Data  Base  Structure 


The  Probe  DB  allows  for  the  study  of  system  changes  over  time  (i.e., 
DRT  scores  as  a  function  of  a  Probe  repetition) .  In  order  to  provide 
the  capability  of  studying  DRT  scores  as  a  function  of  other  variables 
for  any  set  of  DRT  systems,  the  Generic  Probe  Data  Base  (GPRB)  was 
devised.  Its  structure  is  identical  to  the  Expanded  Probe  DB  and  is 
identified  by  the  first  system  number  9600.  The  analysis  routines  of 
the  MENU  package  have  been  modified  to  recognize  a  GPRB  DB  and  treat 
the  systems  accordingly. 

The  analyst  can  design  a  test  series  that  varies  a  single  parameter  or 
pick  from  existing  systems  in  the  overall  DB  to  generate  a  GPRB  volume 
ordered  on  whatever  parameter  he  wishes  to  study.  The  program  GENDB 
(see  Chapter  5)  allows  the  analyst  to  interactively  build  a  GPRB 
sequential  •analysis  volume.  The  probe  analysis  routines  of  MENU  are 
then  used  for  study  of  the  data.  Since  no  Probe  systems  can  have  more 
than  one  version,  the  9600  systems  are  referenced  by  the  systems 
numbers  9600  to  9899.  The  software  has  been  modified  to  use  the 
original  system  numbers  for  all  labels  and/or  figure  headers. 


2.2  DRT  DATA  BASE  BACKUP  PROCEDURES 


Diagnostic  Rhyme  Test  raw  data  files  and  the  DRT  Data  Base  used  for 
comparative  analysis  require  separate  backup  steps.  These  procedures 
have  been  automated  within  the  login  DRT/BACKUP.  By  properly  answering 
all  questions  and  by  loading  DLO:,  DLl;,  DL2:,  or  DL3:  with  the  proper 
RL01/RL02  backup  disk,  both  the  raw  data  files  and  the  DRT  Data  Base 
can  be  backed  up  or  restored  easily.  Backup  volumes  for  all  DRT  Data 
Bases  and  the  .ZFY  raw  data  files  reside  in  the  Disk  Storage  cabinets 
of  the  RADC/EEV  Speech  Processing  Facility  at  Building  1120. 
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CHAPTER  3 


DIAGNOSTIC  RHYME  TEST  SCORING 


The  program  DRT  is  used  for  scoring  input  data  from  DRT  test  sessions. 
It  has  been  described  in  detail  in  numerous  reports  (Ref.  7) .  The 
input  data  is  transmitted  from  the  Radio  Shack  TRS-80  Model  100 
Portable  Computers,  used  by  a  DRT  listener  crew  for  the  entry  of  DRT 
word  choices,  to  a  data  file  located  on  the  PDP-11/44  DRT  account  by 
the  program  MODEL  (Ref.  5).  The  data  files  are  named  in  the  form 
NNNN.ZFY;W,  where  NNNN  will  be  the  previously  assigned  system  number 
and  W  will  be  a  version  that  distinguishes  more  than  one  testing  of 
the  same  system  audio  tape. 

The  program  DRT  takes  data  stored  in  NNNN.ZFY  files,  processes  it  into 
reports,  and  if  requested,  stores  the  processed  data  in  the 
appropriate  updated  Data  Base  Disk.  The  user  specified  inputs  required 
for  this  software  module  include  the  particular  .ZFY  file  and  its 
version.  The  other  input  file  required  by  DRT  resides  on  the  pseudo 
device  DF;  and  is  called  COMKEY.KEY  which  contains  the  DRT  wordlist 
keys.  If  scoring  interrupted  systems,  DRT  requires  that  the  input  file 
NNNN. KEY,  which  contains  a  missing  word  key  for  the  interrupted 
system,  reside  on  the  psuedo  device  DF: . 

DRT  produces  two  optional  reports.  The  first  report  consists  of: 
individual  listener  feature  scores  averaged  across  speakers, 
individual  speaker  feature  scores  averaged  across  listeners,  and  final 
feature  scores  averaged  across  listeners  and  speakers.  The  second 
report  is  a  listing  of  the  "reordering"  process  to  reduce  the  number 
of  listeners  to  a  base  set  of  eight.  This  report  includes  a  matrix  of 
listener  by  speaker  total  scores  averaged  across  features  and  states 
(present  and  absent) ;  listener  deviation  from  the  speaker  means  and 
variance  in  this  deviation;  a  multivariate  F-statistic  analysis  for 
outliers;  and  a  list  of  listeners  reordered  based  on  the  largest 
variance.  DRT  also  allows  the  user  to  add  the  current  system  to  the 
appropriate  DB  disk.  The  routine  checks  for  certain  problems  such  as 
extremely  low  scores  that  may  indicate  a  wrong  wordlist  key  given  and 
allows  the  user  to  correct  it  before  the  addition  to  the  DB. 
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During  the  current  contract  period  several  modifications  have  been 
made  to  the  DRT  program.  The  following  sections  will  discuss  these 
changes.  The  source  code  for  this  latest  version  of  the  DRT  program  is 
available  from  RADC/EEV. 


3.1  DRT  MODIFICATION  FOR  STANDARD  LISTENER  SET 


The  subroutine  MATR  was  modified  to  standardize  when  a  base  set  of 
listeners  is  calculated.  This  is  the  routine  that  reduces  or  "culls" 
the  number  of  listeners  for  a  system  to  a  standard  set  of  eight.  MATR 
calculates  listener  and  speaker  results  averaged  across  all  features 
and  states  along  with  deviations  of  these  results  from  speaker  means 
across  listeners.  MATR  then  takes  these  deviations  and  calculates  the 
variance  across  speakers  and  an  F-statistic  for  each  listener.  MATR 
then  reduces  the  number  of  listeners  to  a  base  set  of  eight  by 
reordering  those  listeners  having  the  largest  variance  at  the  end  of 
the  list.  This  process  was  previously  optional,  it  is  now  standard 
with  the  report  of  the  process  being  optional. 


3.2  DRT  MODIFICATION  FOR  EXPANDED  PROBE  DB 


The  final  modification  to  DRT  was  the  revision  of  code  to  handle  the 
addition  of  Probe  systems  to  a  new  "expanded"  probe  disk.  Original 
probe  disks  were  set  up  just  like  system  disks  except  that  the  system 
numbers  ranged  from  9000  to  9099  meaning  that  the  probe  disk  could 
have  100  systems  with  up  to  200  total  repeat  tests  of  those  100 
systems.  Yet,  all  probes  are  the  same  system  run  every  test  session, 
so  there  was  no  real  need  to  have  higher  versions  of  the  same  system. 
Since  this  fact  was  evident,  the  introduction  of  "expanded"  probe 
disks  came  along  and  their  structure  will  be  discussed  further  in 
Chapter  3.  The  subroutine  ADDDB  was  the  only  subroutine  in  the  DRT 
program  to  undergo  changes  due  to  this  modification.  ADDDB  was 
modified  to  handle  the  addition  of  probe  systems  to  the  DB  differently 
from  the  way  data  systems  are  added.  Instead  of  checking  for  new 
versions  or  multiple  versions  of  a  system,  it  just  finds  the  last 
system  number  on  the  data  base,  increments  the  system  number,  and 
transfers  the  data  in  the  appropriate  space.  The  DB.INX  file  of  the 
"expanded"  probe  data  base  (discussed  further  in  Chapter  3  )  is  one  of 
the  major  differences  between  the  original  probe  disk  and  the  expanded 
probe  disk. 


3.3  DRT  MODIFICATION  FOR  INTERRUPTED  SYSTEMS 


The  DRT  task  used  for  scoring  raw  input  data  for  DRT  sessions  has  been 
revised  to  permit  the  scoring  of  "interrupted"  systems  in  which  known 
DRT  words  are  not  presented  to  the  listeners  due  to  disruptions  of  the 
communications  channel.  Percentage  correct  scores  are  generated  on  the 
basis  of  the  number  of  test  items  actually  presented  to  the  listeners. 
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A  decision  as  to  whether  test  items  are  present  or  not  must  be  made  by 
an  administrator  and  recorded  in  a  separate  file  DF:NNNN.KEY,  where 
NNNN  is  the  DRT  system  number.  The  development  approach  to  Interrupted 
DRT's  and  its  verification  are  covered  fully  in  Chapter  6. 

When  the  DRT  task  begins,  the  user  is  asked  if  he  wishes  to  process  a 
regular  DRT,  an  interrupted  DRT,  or  exit  the  program.  Once  the  user 
answers  the  question,  the  routine  continues  and  branches  to  its 
appropriate  calls.  During  the  execution  of  interrupted  systems,  counts 
of  the  number  of  test  items  for  each  of  the  Subfeature  by  P/A 
interactions  is  maintained  based  upon  the  data  from  the  NNNN. KEY  file. 
Instead  of  always  dividing  the  number  of  correct  responses  (corrected 
for  guessing)  by  a  fixed  constant  as  is  done  for  regular  DRT  systems, 
the  count  of  items  actually  presented  serves  as  the  divisor  when 
creating  a  percentage  correct  score.  Since  interrupted  DRT  systems  are 
added  to  the  DRT  data  base  and  scored  in  a  similar  manner  as  regular 
DRT's,  a  need  for  distinguishing  between  the  two  types  of  DRT  systems 
and  a  location  to  store  these  count  values  was  necessary.  Therefore, 
DRT  was  revised  to  handle  the  addition  of  an  interrupted  system  flag. 
This  interrupted  system  flag  allows  the  analysis  program  to  recognize 
what  type  of  system  it  is  working  with.  It  is  either  set  to  -1  if  the 
system  was  an  interrupted  DRT  or  not  -1  for  a  regular  DRT.  The 
structure  of  interrupted  systems  in  the  DB  is  discussed  further  in 
Chapter  2 . 


3.3.1  Procedures  For  Identifying  Missing  DRT  Words 

Processed  DRT  tapes  are  monitored  with  headphones  in  a  quiet 
environment.  Words  containing  errors  can  be  played  back  numerous 
times.  Due  to  the  nature  of  the  DRT,  being  a  first  consonant  dependent 
test,  the  decision  of  whether  the  word  is  good  or  obliterated  is 
defined  by  whether  the  first  consonant  is  obliterated.  In  most  cases 
this  decision  comes  easily  as  the  entire  word,  or  sometimes  a  string 
of  words,  is  completely  wiped  out,  either  by  errors  or  by  the  mutes 
that  occurred  directly  after  large  concentrations  of  errors.  In  the 
cases  of  words  that  remain  partially  intact,  playback  is  repeated  more 
often.  If  an  audible  artifact,  one  not  easily  attributed  to  processor 
under  test,  is  detected  in  the  first  consonant,  the  word  is  considered 
obliterated.  The  channel  errors  seen  over  these  communications  systems 
caused  large,  momentary  spikes  in  level  making  them  easy  to 
differentiate  from  the  anomalies  of  most  processor  algorithms.  Words 
that  have  an  intact  first  consonant,  even  if  errors  are  within  the 
remainder,  are  considered  good.  Words  considered  to  be  at  zero 
intelligibility  are  marked  off  on  a  DRT  test  booklet  (RADC  Form  108) 
and  the  number  representing  where  they  fall  on  the  pt’ge  (1-58)  is 
marked.  These  forms  are  then  used  to  generate  interrupt  keys  for  the 
scoring  procedure  using  the  program  INTKEY. 
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Interrupted  Word  Key  -  A  utility  INTKEY  was  implemented  to  provide  a 
means  of  generating  interrupt  key  files,  NNNN.KEY,  needed  for  the 
program  DRT.  The  program  INTKEY  begins  by  requesting  the  user  to  enter 
the  number  of  the  file  on  DF;  for  which  this  routine  is  creating  a  key 
and  the  number  of  speakers  in  the  system  to  be  scored.  Since  each  DRT 
booklet  has  four  pages  with  58  words  on  a  page,  the  program  INTKEY 
requests  the  missing  word  number(s)  1  to  58  one  page  at  a  time.  The 
user  is  allowed  to  repeat  the  page  he  is  currently  on  at  any  time  by 
hitting  a  control  Z  which  will  wipe  out  what  was  entered  previously 
for  that  page.  This  process  of  entering  numbers  page  by  page  continues 
until  the  number  of  speakers  entered  by  the  user  at  the  start  of  the 
routine  is  reached.  Once  the  user  enters  all  the  required  data  for  one 
speaker,  he  is  asked  if  he  wishes  to  accept  that  speaker.  When  all 
speakers  are  entered  and  accepted,  the  program  INTKEY  prints  out  a 
table  showing  the  choices  the  user  had  made.  An  example  of  the  output 
listing  from  executing  INTKEY  is  shown  in  Figure  3.1.  The  source  code 
for  INTKEY  is  available  from  RADC/EEV. 


INTERRUPTED  WORD  KEY  FILE  4024. KEY 
TOTAL  %  MISSING  WORDS  13.5057 
SPEAKER  NO.  1  %  MISSING  WORDS  13.3621 

1234567890123456789012345678901234567890123456789012345678 
0000111000000011100001110000000000011100000000001110000000 
0011100000000000000001110000000010010000000000000000000000 
0000000000000000000000000000000000000000000111110000000000 
0000000000010110000000000000000000000000000000000000000000 
1234567890123456789012345678901234567890123456789012345678 
SPEAKER  NO.  2  %  MISSING  WORDS  12.5000 

1234567890123456789012345678901234567890123456789012345678 
0111000000000011100001110000000000000000000011100000000000 
0000000000011110000000000000000000000000000000000000000000 
OOOOOOOOOOOOOOOOOOOOOllllOOOOOOOOOOOlllOOOOOOOOOOOOOOOOOOO 
0000000000000000000000111000000011100000000000000000000000 
1234567890123456789012345678901234567890123456789012345678 
SPEAKER  NO.  3  %  MISSING  WORDS  14.6552 

1234567890123456789012345678901234567890123456789012345678 

0111000000000001110000000000000000011100000000000000000000 

0000000000000000000000111000000000000000000011100000000000 

0000000000000000000000000000000000000000000000111111000000 

0001111000000000001111000000000000011111000000000000000000 

1234567890123456789012345678901234567890123456789012345678 

Figure  3.1 

Example  of  INTKEY  program  output 
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CHAPTER  4 


DIAGNOSTIC  RHYME  TEST  COMPARATIVE  ANALYSIS  SYSTEM 


Comparative  analysis  of  DRT  results  is  available  through  a  software 
package  called  MENU.  With  this  package,  an  analyst  is  able  to  access 
all  DRT  systems  that  are  stored  on  Data  Base  volumes.  This  is  a 
menu-driven  program  which  provides  the  capability  to  select  various 
options  as  shown  in  the  main  menu  listed  below. 

1  =  DRT  PRINTOUT  OF  COMPLETE  RESULTS 

2  =  DRT  SUMMARY  PRINTOUT 

3  =  PLOT  DRT  PROFILE 

4  =  ANALYSIS  OF  VARIANCE  2  TO  9  SYSTEMS 

5  =  STUDENTS 'S  T-TEST  FOR  2  SYSTEMS 

6  -  PROBE  ANALYSIS 

7  =  LIST  DATA  BASE 

8  =  LIST  RAW  DATA 

9  -  ENTER  AND  PLOT  ***  DAM  ***  DATA 

10-  CULL  LISTENERS 

11-  AVERAGE  ACROSS  SYSTEMS  2  TO  9  SYSTEMS 

12-  MERIT  LISTENERS 

0  -  EXIT  MENU 


Most  of  the  twelve  programs  have  been  described  in  previous  reports 
(Ref.  7).  New  options  and  modifications  to  existing  options  will  be 
described  in  this  chapter.  A  maximum  of  two  DRT  Data  Bases  each  with 
up  to  100  systems  can  be  accessed  by  MENU.  Each  DB  is  defined  by  its 
three  files  (DB.INX,  DB.HDR,  DB.DAT)  and  must  be  mounted  on  pseudo 
device  DK2 :  or  DK3 : .  Details  of  the  DB  file  structure  are  given  in 
Chapter  2  of  this  report.  Output  files  from  the  various  options  are 
identified  by  the  .DMP  extension  and  are  normally  spooled  to  the 
printer  and  deleted.  Options  3,  6,  and  9  produce  data  plots  and 
require  the  TEKTRONIX  4015-1  terminal  located  at  the  Speech 
Laboratory.  The  source  code  for  this  latest  version  of  the  MENU 
package  is  available  from  RADC/EEV. 
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4.1  GLOBAL  MODIFICATIONS 

A  number  of  revisions  were  made  to  the  MENU  package. The  revisions  were 
motivated  by  the  need  to  remove  existing  "bugs"  and  the  desire  to  add 
new  capabilities  to  the  package.  Some  of  the  changes  extend  to  all  of 
the  tasks  making  up  the  MENU  package  while  other  changes  are  task 
specific . 

The  MENU  package  previously  consisted  of  one  overlayed  task  with 
separate  analysis  options  implemented  as  subroutines.  Due  to  the 
growth  of  the  MENU  package  and  potential  future  additions ,  it  was  not 
cost-effective  to  maintain  the  menu  implementation  as  a  single 
overlayed  task.  It  has  since  been  broken  into  a  single  executive  task 
MENU  . . .  which  spawns  the  subtasks  to  perform  one  of  the  specific 
menu  options.  Thus  each  subtask  can  use  a  full  16-bit  virtual  address 
space . 

The  development  of  the  new  MENU  package  was  performed  on  the  DRTDEV 
virtual  disk  at  [200,222]  with  the  final  executable  files  placed  on 
[200,200].  An  object  library  file,  [ 200 , 222 ]MENULIB ,  was  created  to 
contain  all  object  modules  required  to  build  the  MENU  tasks.  All  of 
the  tasks  are  altered  to  do  some  of  the  things  previously  done  in  the 
root  section  of  the  MENU  task  such  as  assigning  LUN's  to  a  listing 
file  or  to  the  graphics  terminal,  calling  SYSSEL  to  select  systems  for 
analysis,  and  controlling  program  flow. 

The  MENU  package  was  also  revised  to  permit  the  analysis  of 

"interrupted"  DRT  data.  Comparisons  between  interrupted  and 
non- interrupted  systems  is  possible  as  all  data  is  ultimately 

represented  by  a  percentage  correct  score.  The  revision  called  for  a 
larger  data  structure  to  contain  the  percentage  scores.  The  GETSXL 
subroutine  was  changed  to  detect  interrupted  systems  and  to  create 
percent  correct  scores  on  the  basis  of  test  items  presented.  The  STATS 
subroutine  was  changed  so  as  not  to  convert  statistics  to  percentage 
scores,  but  instead  expects  the  data  passed  to  it  to  already  be  in  a 
percentage  format.  In  all  output  listings  generated  by  MENU, 

interrupted  systems  are  now  indicated  by  the  presence  of  '/I'  appended 
to  the  system's  identification  number. 

Another  modification  to  the  MENU  package  was  the  introduction  of 
speaker  subsets.  The  first  five  options,  PRINT,  SUMARY,  PLOT,  ANOVA, 
and  TTEST,  allow  the  user  to  choose  if  they  would  like  speaker 

subsets.  What  is  meant  by  speaker  subsets  is  that  in  systems  with  six 
speakers,  the  user  can  choose  to  use  the  first  or  last  three  speakers 
or  all  speakers  when  executing  any  of  the  first  five  options.  When 
working  with  three  speaker  systems  and  the  user  chooses  to  do  speaker 
subsets,  the  routine  will  print  a  message  to  the  user  and  default  to 
the  first  three  speakers.  In  order  to  be  able  to  handle  this  new 
option,  a  new  subroutine  called  SYSTEM  replaced  the  old  subroutine 
SYSSEL  because  SYSTEM  not  only  handles  the  entry  of  the  system  number^ 
and  versions  but  also  handles  the  input  of  what  speakers  to  include  in 
the  analysis  option  chosen. 
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In  response  to  any  prompt  in  the  MENU  package,  the  user  may  enter  a 
control  Z  which  will  then  go  through  a  sequence  of  steps  before 
terminating  the  current  task.  A  control  Z  sequence  will  close  all 
files  opened  for  output  and  stop  once  this  is  done  where  as  before  a 
control  Z  entered  in  response  to  a  prompt  would  terminate  and 
sometimes  lock  files  used  to  execute  the  program.  This  modification  of 
a  control  Z  sequence  affected  mostly  all  routines  in  the  MENU  package 
and  this  sequence  should  be  followed  in  every  new  addition  to  the 
analysis  package. 

Interrupted  DRT  systems  are  now  recognized  in  the  output  listings  of 
the  MENU  package.  All  system  identification  numbers  for  interrupted 
systems  are  postfixed  with  "/I"  when  displayed  in  any  of  the  various 
subroutine  outputs  of  MENU.  All  options,  except  XDAM,  were  affected  by 
this  modification. 

When  being  prompted  for  the  "number  of  listeners"  in  the  subroutine 
QUERY  and  the  user  has  selected  the  option  to  enter  a  specific  number 
of  listeners  but  has  selected  a  number  which  is  out  of  range,  an  error 
message  is  displayed  and  the  user  is  given  another  chance  to  enter  the 
number  of  listeners. 


4.2  OPTION  MODIFICATIONS 

The  PRINT  option  was  changed  to  output  its  listings  of  processed  data 
in  a  different  order.  The  request  for  a  complete  printout  will  result 
in  a  multi -page  report  structured  as  follows: 

1.  a  page  for  each  listener  giving  mean  scores  and  standard  errors 
across  speakers  for  all  attributes. 

2.  a  page  for  each  speaker  giving  mean  scores  and  standard  errors 
across  listeners  for  all  attributes. 

3.  a  page  giving  mean  scores  and  standard  errors  across  listeners 
and  speakers  for  all  attributes. 

4.  a  matrix  of  listener-speaker  scores. 

5.  a  matrix  of  listener  attribute  scores. 

The  request  for  an  abbreviated  version  will  produce  items  2,  3,  4,  and 
5  above . 

The  PROBE  option  in  MENU  was  revised  to  accept  the  new  GENERIC  PROBE 
Data  Bases  with  systems  in  the  9600-9899  range  and  to  change  the 
graphic  display  labels  accordingly  for  these  special  data  bases.  With 
this  addition,  the  routine  can  be  used  to  provide  functional  plots  of 
intelligibility  vs  experimental  parameter.  PROBE  provides  plots  of  DRT 
probe  scores  versus  probe  repetitions  which  are  useful  in  tracking 
listener  performance.  These  plots  are  also  used  for  soundroom 
equipment  evaluation  so  that  problems  with  equipment  can  be  found  and 
corrected.  There  are  several  plot  options  within  PROBE  that  allow  the 
user  to  interactively  select  the  data  that  best  demonstrates  what  he 
is  looking  for.  Since  a  Probe  DB  can  contain  up  to  300  Systems,  a 
movable  window  with  a  maximum  size  of  100  can  be  positioned  within  the 
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Probe  DB.  Several  modifications  were  made  to  PROBE  concerning  the 
display  of  output.  One  such  change  was  the  scaling  and  centering  of 
the  output  for  maximum  resolution  when  less  than  20  systems  are  being 
displayed.  Other  changes  involved  checking  for  missing  speakers  and 
cleaning  up  the  display's  header  format. 

Other  plot  options  in  PROBE  are:  the  selection  of  a  specific  speaker 
or  the  sum  over  all  speakers;  the  selection  of  a  specific  listener,  an 
average  of  the  standard  set  of  8  listeners,  or  an  average  of  all 
listeners;  the  selection  of  a  specific  attribute  or  the  sum  over  all 
attributes;  the  selection  of  the  present  (p) ,  absent  (a),  or  combined 
(c)  state  of  the  attribute  chosen;  the  selection  of  a  composite  plot 
with  p/a/c  states  all  plotted;  and  a  linear  regression  fit  to  the 
listener,  attribute,  and  state  data  chosen. 

The  p/a/c  plots  for  a  specific  listener  include  the  group  performance 
(less  the  specified  listener)  with  the  95%  confidence  intervals,  the 
listener  performance,  and  information  on  the  listener  attendance  and 
performance  relative  to  the  group.  The  linear  regression  plots  show 
the  listener  performance,  a  linear  fit  to  that  data,  and  a  listing  of 
the  regression  coefficients. 

The  TTEST  option  was  also  revised  to  test  the  significance  of  T-values 
made  at  alpha  =  .01  as  well  as  at  alpha  =  .05  depending  on  what  the 
user  specifies.  If  significant  mean  differences  exist  TTEST  marks 
"*"'s  next  to  the  scores.  The  format  of  the  output  listing  for  TTEST 
was  changed  to  allow  for  the  display  of  these  significant  mean 
differences  in  all  the  tables.  Also  tests  between  two  systems  for  only 
one  user  specified  speaker  or  speaker  subsets  is  now  possible. 

The  XDAM  option  was  modified  to  handle  the  plotting  of  new  descriptors 
due  to  a  change  in  the  information  contained  in  DAM  test  results 
received  from  Dynastat.  Scaling  of  the  XDAM  plot  waj  needed  in  order 
to  include  these  new  descriptors  and  larger  arrays  were  made  to  handle 
the  storage  of  the  input  of  these  values.  A  new  subroutine,  CHECK,  was 
added  to  XDAM  in  order  to  allow  the  user  to  review  and  change  any  of 
the  data  he  entered  which  might  happen  to  be  incorrect. 

The  RAWDAT  option  was  revised  to  display  test  item  counts  along  with 
the  raw  data  of  the  system  number  requested  when  an  interrupted  DRT 
system  is  encountered.  Lastly,  the  DBLIST  option  now  generates 
automatic  paging  of  its  listings. 
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4.3  NEW  OPTIONS 


4.3.1  Analysis  Of  Variance  Algorithm 

A  revised  Analysis  of  Variance  model  was  researched  and  implemented 
for  use  in  the  statistical  analysis  package  of  DRT  test  data.  The 
experimental  design  previously  established  for  the  DRT  MENU  package 
was  used  as  a  guide  for  the  development  of  the  new  ANOVA  model.  This 
multi -dimensional  model  provides  a  means  for  the  testing  of 
significant  System,  Speaker,  and  Feature  effects  on  DRT  scores,  as 
well  as  significant  "interaction"  effects.  The  new  model  also  takes 
into  consideration  the  fact  that  the  Speaker,  and  the  Feature  factors 
are  "Repeated  factors"  meaning  that  each  subject  for  a  particular 
system  is  presented  with  DRT  stimuli  for  each  and  every  possible 
combination  of  speaker  and  feature  included  in  the  experiment.  The 
System  factor,  however,  is  not  considered  a  repeated  factor  since  the 
experimental  procedure  does  not  demand  that  the  same  group  of  subjects 
be  presented  with  the  stimuli  from  all  of  the  possible  systems  present 
in  the  study.  The  advantage  of  the  repeated  measure  design  lies  within 
the  fact  that  since  each  subject  responds  to  stimuli  representing  all 
levels  of  a  particular  factor  (e.g.  Feature),  there  is  no  confounding 
of  the  source  of  response  variation  between  the  'treatment'  factor 
(e.g..  Feature)  and  ' Between- subject'  differences  in  response 
tendencies.  The  speech  stimuli  can  be  categorized  on  three  important 
dimensions : 

Feature- -phonetic  attributes  of  the  stimuli 

Speaker- -who  spoke  it 

System  --what  type  of  communication  system  processed  the 
stimuli  before  its  presentation  to  the  subjects 

The  variability  in  the  ability  of  the  subjects  to  identify  these 
stimuli  as  reflected  in  the  DRT  intelligibility  scores  can  be  assumed 
to  be  attributable  to  these  three  factors  - -Feature ,  Speaker,  and 
System.  The  ANOVA  technique  provides  a  means  of  testing  hypotheses 
concerning  the  variability  seen  in  these  scores  (e.g.,  the  hypothesis 
that  the  variation  in  DRT  scores  between  systems  is  no  different  than 
the  variation  of  scores  seen  for  stimuli  all  processed  by  the  same 
system) . 

The  ANOVA  model  construct  can  be  represented 
(B  X  C  X  S/A)  X  A 

where  A  =  System  factor 
B  -  Speaker  factor 
C  =  Feature  factor 

S/A  =  Subjects  nested  within  System  factor. 


Note  also  that 

the  term  in  parentheses  indicates  that 

the 

included 

factors  are 

repeated  measures.  A  table  of  the  sources 

of  response 

variations,  their  degrees  of 

freedom,  the  computational 

formula  used 

to  determine  their  Sums  of  Squares  (SS)  values  used  in  estimating  mean 

variances,  and 

the  error  terms  used  in  hypothesis  testing 

concerning 

these  sources 

are  provided  in 

Table  4.1. 

Source 

df 

SS  computation 

Error  Term 

****** 

** 

************** 

-kic'k'k'k'k'k'ic'k'k 

System  (A) 

a- 1 

[A]  -  [T] 

S/A 

Speaker  (B) 

b-1 

[B]  -  [T] 

B 

x 

S/A 

Feature  (C) 

c-1 

[C]  -  [T] 

C 

X 

S/A 

Sys  X  Spk 

(a-l)(b-l) 

[AB]-[A]-[B]+[T] 

B 

X 

S/A 

Sys  X  Feat 

(a-l)(c-l) 

[AC1-[A]-[C]-H[T] 

C 

X 

S/A 

Spk  X  Feat 

(b-a)(c-l) 

[BC]-[B]-[C]+[T] 

B 

X 

C  X  S/A 

Sys  X  Spk  X 

(a-l)(b-l)(c-l) 

[ABC] - [AB] - [AC] - [BC]+ 

B 

X 

C  X  S/A 

Feat 

[A]+[B]+[C]-[T] 

Subj  ects/Sys 

a(n-l) 

[AS]  -  [A] 

Spk  X  Subj/Sy 

s  a[(n-l)(b-l) 

]  [ABS]-[AS]-[AB]+[A] 

Feat  X  Subj/Sys  a[(n-l)(c-l) 

]  [ACS]-[AS]- [AC]+[A] 

Feat  X  Spk  x 

a[(n-l)(b-l) 

[ABCS] - [ABS] - [ACS] - [ABC] 

+ 

Subj /Sys 

(c-1)] 

[AB]+[AS]+[AC] - [A] 

a  number  of  Systems 
b  =  number  of  Speakers 
c  =  number  of  Features 
n  =  number  of  Subjects  per  System 

*************************************************************** 
Definition  of  terms  in  SS  computation  formulas  are  explained 
in  Geoffery  Koppel's  Design  and  Analysis  (Ref.  8  pp.  191-195). 

Table  4.1 

The  significance  of  all  main  effects  and  interaction  effects  is 
determined  by  comparing  the  obtained  F-ratio  with  a  critical  F  value 
which  represents  a  lower  bound  of  expected  F- ratios  based  upon  the 
assumption  that  the  actual  population  means  under  analysis  are  equal 
(i.e.,  the  null  hypothesis  is  true)  and  the  probability  of  obtaining 
an  F-value  equal  to  or  larger  than  this  critical  F  value  is  at  a  known 
level.  This  probability  level  is  referred  to  as  the  alpha  level  and 
represents  the  level  of  'risk'  that  the  researcher  is  willing  to  take 
in  falsely  rejecting  the  null  hypothesis  and  concluding  that  there  are 
significant  mean  differences  when  in  fact  the  population  means  are 
equal.  The  critical  F  value  for  a  given  alpha  level  and  degrees  of 
freedom  parameters  can  be  obtained  from  the  appropriate  sampling 
distribution  of  the  F  statistic  and  is  readily  available  via 
statistical  tables. 
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The  F-ratio  formed  by  the  estimator  of  the  variability  of  scores 
between  systems  (i.e.,  mean  squares  for  System  factor)  and  the 
estimator  of  the  variability  of  scores  within  systems  (i.e.,  mean 
squares  for  'subjects  nested  in  systems',  Subj/Sys)  can  be  used  to 
test  the  hypothesis  that  the  mean  difference  between  systems  is  zerc . 
That  is,  this  statistic  can  be  used  to  determine  whether  the  systems 
differ  statistically  in  their  overall  intelligibility  scores.  Upon  the 
detection  of  significant  System  differences,  a  more  detailed  analysis 
to  pinpoint  which  particular  system  differs  from  others  is  made  using 
the  "pairwise  mean  comparison"  technique  known  as  the  Newman-Kuels . 

The  pairwise  comparison  analysis  is  called  only  if  the  System  main 
effect  is  significant.  The  formula  used  to  determine  the  critical  mean 
difference  (C.D.)  between  a  pair  of  means  is: 

C.D.  =  q  (r,df)  *  sqrt(MSerr  /  b*c*n) 

where  q  Studentized  Range  Statistic  determined  by  the 
parameters  r,  df,  and  the  alpha  level  chosen 
r  =  the  number  of  'steps'  between  the  pair  of  means 
being  compared 

df  =  the  degrees  of  freedom  of  the  System  Factor's 
error  term  (i.e.,  Subj/Sys) 

MSerr  -  the  Mean  Square  of  the  System  Factor's  error  term 
b  =  the  number  of  speakers 
c  “  the  number  of  features 
n  “  the  number  of  subjects  per  system 

Note  that  the  critical  mean  difference  used  to  compare  a  pair  of  means 
will  depend  on  the  number  of  means  located  between  the  pair.  The 
Studentized  Range  Statistic  required  for  a  particular  comparison  is 
evaluated  at  a  specific  alpha  level  and  is  realily  available  via 
statistical  tables. 

The  System  by  Feature  interaction  term  is  used  to  determine  if  a 
pattern  of  Feature  scores  is  consistent  across  all  individual  systems 
included  in  the  study.  If  the  System  by  Feature  interaction  term  is 
significant,  the  evaluation  of  the  simple  main  effects  of  the  System 
factor  at  each  level  of  the  Feature  factor  is  carried  out.  That  is, 
the  differences  across  all  systems  for  each  of  the  particular  speech 
attributes  is  tested  for  significance  in  an  attempt  to  clarify  how  the 
Feature  scores  differ  from  system  to  system.  A  separate  error  terra  is 
calculated  for  each  simple  main  effect  which  uses  only  the  scores 
involved  at  the  Feature  level  for  which  the  effect  is  being  evaluated. 
For  a  discussion  of  the  rational  for  using  the  separate  error  terms 
for  each  simple  main  effect  as  compared  to  a  more  'heterogeneous' 
approach  to  the  selection  of  error  terms  as  prescribed  by  Winer  (Ref. 
9)  see  Koppel's  Design  and  Analysis  (Ref.  8  pg.  442). 
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The  numerator  term  for  the  F-ratio  used  to  evaluate  this  simple  main 
effect  at  a  given  Feature  level  is  derived  using  the  following  formula 
to  determine  the  sums  of  squares; 


SS. 


I  (Z 

1-1  Iff"!  j"l 


■’A  at  Cj,  nb 

with  degrees  of  freedom  -  a  -  1. 
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i-1 
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where 


i  -  index  over  Systems 
j  -  index  over  Speakers 
k  “  index  over  Features 

m  -  index  over  Subjects  (Listeners)  per  System 
d  -  experimental  data  point  at  i,j,k,m 


The  F-ratio  denominator  term  is 


anb  anb 

Z  Z  (Zd)^  Z  (Z  Zd)^ 

1-1  0-1  J-l  l-l  lff-1  J-1 

^^S/A  at  C^  “  b  ■ 

with  a  *  (n-1)  degrees  of  freedom. 

The  System  by  Speaker  interaction  term  is  used  to  determine  if  a 
pattern  of  Speaker  scores  is  consistent  across  all  individual  systems 
included  in  the  study.  If  the  System  by  Speaker  interaction  term  is 
significant  the  evaluation  of  the  simple  main  effects  of  the  System 
factor  at  each  level  of  the  Speaker  factor  is  carried  out.  That  is, 
the  differences  across  all  systems  for  each  of  the  particular  speakers 
is  tested  for  significance  in  an  attempt  to  clarify  how  the  Speaker 
scores  differ  from  system  to  system. 

The  F-ratio  used  to  evaluate  this  simple  main  effect  for  a  given 
Speaker  is  derived  in  the  same  manner  as  used  for  the  Feature  effect 
across  Systems.  For  the  Speaker  effect  all  occurences  of  the  indicies 
"j"  and  "k"  are  s’.apped  in  the  preceding  equations. 

ANOVA  Software  Implementation  -  The  program  ANOVA  has  been  developed 
to  perform  the  required  computations  for  the  new  Analysis  of  Variance 
of  DRT  systems  in  the  comparative  analysis  DB.  It  replaces  the 
previous  ANOV  routine  and  can  be  found  on  the  virtual  disk  DRTDEV  at 
UIC  [200,200].  The  user  specified  inputs  required  for  this  software 
module  include  the  number  of  systems  to  compare,  the  particular 
systems  in  the  data  base  to  compare,  the  number  of  subjects  to  be 
included,  the  speakers  or  speaker  subset  to  be  included  and  the  alpha 
level  at  which  to  mark  significant  differences.  Previously  written 
subroutines  that  access  the  data  base  were  retained.  The  output  of 
this  module  consists  of: 


1.  A  vector  of  means  for  each  Main  effect  (i.e.,  System, 
Speaker,  Feature). 

2.  A  matrix  of  means  for  each  Interaction  effect  (e.g.,  Sys  x 
Feat) . 

3.  An  Analysis  of  Variance  Table  listing  for  each  source; 

1.  the  sum  of  the  square  deviations 

2.  the  degrees  of  freedom 

3.  the  mean  square  deviation 

4.  the  F  ratio  and  indication  of  significance. 

5.  the  critical  F  ratio  value  from  a  lookup  table 

4.  If  the  System  main  effect  is  significant,  the  results  of  a 
multiple  pairwise  comparison  of  the  System  means  usi^.g  the 
Newman-Kuels  technique  are  displayed  in  the  form  of  a  matrix 
of  mean  differences  for  all  possible  pairs  of  systems 
included  in  the  steady.  Also  included  is  an  indication  as  to 
whether  each  mean  difference  is  statistically  significant. 
This  output  is  provided  by  the  subroutine  NKUELS . 

5.  If  the  System  x  Feature  interaction  is  significant,  an 
analysis  of  the  simple  main  effect  of  Systems  at  each  Feature 
level  is  performed.  A  test  of  significant  system  differences 
for  each  Feature  level  is  made  and  a  display  of  the  Sums  of 
Squares  (SS) ,  Mean  Squares  (MS),  and  the  F  ratio  is  provided 
for  each  simple  main  effect.  Also  displayed  for  each  Feature 
level  is  an  error  term's  Sums  of  Squares  and  Mean  Squares 
values.  This  output  is  provided  by  the  subroutine  SMAINF. 

6.  If  the  Speaker  x  System  interaction  is  significant,  an 
analysis  of  the  simple  main  effect  of  Systems  at  each  Speaker 
level  is  performed.  A  test  of  significant  system  differences 
for  each  Speaker  level  is  made  and  a  display  of  the  Sums  of 
Squares  (SS)  ,  Mean  Squares  (MS),  and  the  F  ratio  is  provided 
for  each  simple  main  effect.  Also  displayed  for  each  Speaker 
level  is  an  error  term's  Sums  of  Squares  and  Mean  Squares 
values.  This  output  is  provided  by  the  subroutine  SMAINS . 


The  F-ratios  obtained  for  the  Main  effects  and  the  interaction  effects 
are  compared  with  a  critical  F-ratio  obtained  from  the  file 
"FDIS.TBL".  This  file  is  formatted  to  perform  as  a  lookup  table  with 
indexing  provided  by  the  degrees  of  freedom  for  the  numerator  and 
denominator  of  the  F-ratio.  The  Studentized  Range  Statistic  required 
by  NKUELS  for  a  particular  comparison  is  looked  up  in  the  file 
"QDIS.TBL".  Both  tables  contain  values  for  alpha  levels  of  0.05  and 
0.01.  These  tables  are  located  on  DKl : [ 200 , 200]  for  use  with  MENU  and 
on  the  virtual  disk  DRTDEV  at  UIC  [200,200]. 


4.3.2  System  Average  Profile 

A  new  option  called  AVGSYS  was  implemented  into  the  DRT  MENU  package 
to  enable  the  analyst  to  obtain  results  averaged  over  several  systems. 
AVGSYS  allows  for  averages  over  two  to  nine  systems.  In  order  to 
accomplish  this,  three  guidelines  must  be  followed.  The  number  of 
speakers,  the  speaker  ID's,  and  the  number  of  listeners  must  be  the 
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same  in  each  of  the  systems  included  in  the  analysis.  Once  the 
guidelines  are  followed,  the  routine  proceeds  to  output  a  report  of 
the  averages.  Along  with  these  averages  by  speaker,  the  report 
displays  the  standard  error  across  listener  by  speaker  interactions 
averaged  across  systems,  what  systems  were  included  in  this  analysis 
along  with  the  header  information,  the  list  of  speakers  in  each 
system,  and  the  number  of  listeners. 


4.3.3  Listener  Merit  Coefficient 

The  PROBE  method  of  tracking  listener  performance  does  not  take  into 
account  how  many  times  a  listener  was  culled  out  of  the  final  set  of 
eight  listeners  used  to  calculate  final  DRT  scores.  Existing  software 
was  revised  to  keep  a  record  of  how  often  individual  listeners  are 
culled  from  the  scoring  process  and  to  take  into  account  whether  they 
are  culled  for  being  above  or  below  the  statistical  thresholds  that 
are  set.  This  new  routine  is  called  MERIT  and  has  been  added  to  the 
DRT  MENU  analysis  package.  The  program  operates  on  a  "window"  of 
systems  in  a  probe  DB.  This  window  is  defined  in  the  same  manner  as  in 
the  PROBE  option. 


The  MERIT  option  outputs  a  table  of  calculations  for  the  analyst  to 
use  to  evaluate  each  listener's  performance.  The  calculations  are; 

1.  The  number  of  times  (a^)  each  listener  L  was  actually 
present  for  the  specified  number  of  systems. 

2.  The  number  of  times  each  listener  was  in  positions  L  through 
12  during  a  "culling"  procedure  similar  to  the  CULL  routine 
in  the  MENU  package. 

3.  The  ratio  of  the  number  of  times  each  listener  was  "culled" 
out  to  the  total  number  of  times  that  listener  was  present. 

4.  A  listener  merit  factor,  D(L),  calculated  over  all  systems 
within  the  analysis  window  for  which  listener  L  was  present. 
The  equation  for  D(L)  is  presented  below. 

For  a  given  listener  L, 


,  *1.  p 

D  -  i  y  — 

a,  4  h  n 


“1 

is 


^  1-1  j-i 


UL 


where 


deviation  from  speaker  j '  s  mean  for  system  i  at  the 
level  which  listener  L  is  culled  from  the  listener  crew 
level  (12,11,  ...  ,1)  at  which  listener  L  is  culled 

from  the  listener  crew  for  system  i 
total  number  of  systems  listener  L  participated  in 
nvimber  of  speakers  in  system  I 
number  of  listeners  in  system  i 
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By  reviewing  all  of  these  calculations  in  the  final  output,  the 
analyst  can  see  which  listener  is  being  culled  out  most  of  the  time  in 
relation  to  the  number  of  times  that  particular  listener  was  present 
and  also  look  at  their  normalized  deviation  to  check  whether  they  were 
always  above  or  below  the  mean.  These  calculations  are  displayed  in  a 
table  format  where  the  analyst  can  observe  them  for  each  listener.  The 
output  is  ordered  in  three  different  ways:  by  number  of  times  present, 
by  listener  identification  numbers,  and  by  the  culling  out  percentage. 
The  user  can  also  observe  where  a  listener's  position  tends  to  be 
after  the  culling  procedure  takes  place. 


MERIT  Software  Implementaion  -  The  program  MERIT  was  developed  to 
provide  a  better  means  of  rating  listener  performance.  MERIT  requires 
either  an  EXPANDED  or  GENERIC  PROBE  DB  at  the  pseudo  device  DK2  or 
DK3 .  If  no  PROBE  disk  was  found  on  either,  it  prints  a  message  and 
returns  to  the  main  menu  in  EXMAIN.  If  a  PROBE  disk  is  present,  the 
routine  displays  the  top  PROBE  system  and  how  many  systems  exist  on 
the  disk.  The  analyst  is  then  asked  where  to  start  the  analysis  and 
how  many  systems  (referred  to  in  the  program  as  "window  size")  to  be 
included  in  this  analysis.  The  analyst  may  request  from  3  to  300 
systems  but  if  the  analyst  does  not  enter  a  valid  response  the  routine 
will  default  to  the  largest  number  of  systems  available.  Note, 
however,  the  analyst  must  enter  the  top  probe  system  number  to  be 
included  because  the  routine  begins  the  processing  at  the  top  system 
number  and  decrements  each  time  until  it  reaches  the  "window  size" 
specified. 

This  routine  permits  the  user  to  get  a  full  or  abbreviated  report  of 
the  processing.  The  full  report  consists  of  a  listing  of  the  listener 
ID's,  the  order  of  the  listeners  after  the  elimination  procedure,  the 
last  variance  at  which  each  listener  was  eliminated,  and  their 
normalized  deviation  for  each  system  included  in  the  analysis.  The 
final  phase  of  this  routine  outputs  three  tables  that  contain  the  same 
information  yet  ordered  in  three  different  ways.  All  three  tables 
display  the  computations  described  in  the  previous  section.  The  first 
table  orders  them  by  the  number  of  times  each  listener  was  present. 
The  second  table  orders  them  by  listener  identification  numbers  and 
the  third  table  is  ordered  by  the  culled  out  percentage.  The 
abbreviated  report  only  generates  the  three  tables.  Lastly,  the 
analyst  is  requested  to  either  choose  a  new  window  size  or  exit  the 
program.  If  a  new  window  size  is  chosen,  the  process  starts  all  over 
again. 
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CHAPTER  5 


DRT  SYSTEM  UTILITY  PROGRAMS 


This  Chapter  will  describe  three  utility  programs  that  are  used  with 
the  RADC/EEV  DRT  System.  The  first  of  these  routines,  ZTILTY,  provides 
for  the  maintenance  of  the  Raw  DRT  Data  Base  while  the  second, 
DBTILTY,  maintains  the  DRT  Analysis  Data  Base.  The  third  program, 
GENDB,  is  a  new  addition  to  the  DRT  System.  This  routine  can  be  used 
to  search  the  Analysis  DB  for  systems  with  common  characteristics. 
GENDB  provides  a  list  of  the  systems  that  may  be  edited  by  the  EDT 
program  and  will  generate  a  "generic  probe  data  base  disk"  that 
contains  the  selected  systems  in  sequential  order.  This  disk  can  be 
used  with  any  of  the  MENU  options.  Specifically,  the  PROBE  and  MERIT 
options  will  provide  special  information  not  previously  available. 


5.1  D^  R^  DB  UTILITY 

The  main  utility  package  used  with  the  DRT  Raw  Data  Base  created  at 
the  USAF  RADC/EEV  Speech  Processing  Facility  is  ZTILTY.  This  package 
is  a  useful  tool  in  the  generation  and  maintenance  of  the  already 
existing  . ZFY  files  by  providing  certain  options  for  the  modification 
of  raw  data  or  header  information  that  was  incorrectly  input.  It  also 
allows  for  the  comparison  of  data  from  two  files  and  dumps  these 
differences  to  the  user-specified  device.  ZTILTY  also  permits  the 
generation  of  a  new  3 -speaker  system  from  a  subset  of  any  three 
speakers  of  a  6 -speaker  system  and  the  generation  of  two  3 -speaker 
systems  from  a  6-speaker.  Those  two  options  have  been  used  extensively 
in  the  past  for  analysis  purposes.  Other  options  included  in  ZTILTY 
concern  the  deleting  of  listener's  data  from  a  .ZFY  file  or  the 
listing  of  file  headers. 
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The  program  resides  within  the  DRT/STAFF  account  on  device  DKl :  and 
can  be  executed  by  the  RUN  ZTILTY  command.  The  initial  menu  of  options 
is : 


1”  FILE  HEADER  LISTINGS 
2=  FILE  DUMP 

3=  FILE  COMPARE  AND  DUMP 
4=  FILE  HEADER  AND  MODIFY 
5=  LISTENER  DELETE 
6-  DATA  MODIFY 
7=  FILE  MERGE 
8=  VOLUME  HEADER  LISTINGS 
9-  THREE  SPEAKER  SUBSET  SYSTEM 
0=  END  ZTILTY 

Several  input/output  files  are  used  by  ZTILTY  and  are  all  expected  to 
be  found  or  created  on  the  default  local  system  device  SY:  at  the 
default  UIC  (i.e.,  [200,200]).  A  description  of  the  files  used  by  the 
options  is  as  follows: 

ZFILE  =  (NNNN.ZFY:W)  MAIN  INPUT  .ZFY  FILE 
XFILE  =  (MMMM.ZFY;W)  SECONDARY  INPUT  .ZFY  FILE 
MFILE  =  (NNNN.M0D:W)  MODIFIED  OUTPUT  .ZFY  FILE 
SPFILE  -  (NNNN.DMP;W)  SPOOLED  OUTPUT  LISTING  FILE 

where  NNNN  and  MMMM  are  DRT  system  numbers;  .ZFY  and  .MOD  files  are 
DRT  output  listing  files;  and  .DMP  file  is  an  ASCII  output  file. 

This  program  has  not  been  modified  during  this  contract  period.  A  full 
description  of  the  program  can  be  found  in  Reference  7. 


5.2  DRT  ANALYSIS  DB  UTILITY 

An  Analysis  Data  Base  utility  program  is  available  through  the  routine 
DBTILTY.  This  utility  program  is  mainly  used  for  data  base 
maintenance.  It  resides  within  the  DRT/STAFF  account  on  device  DKl; 
and  can  be  executed  by  the  RUN  DBTILTY  command.  The  initial  menu  of 
options  is: 


1  -  DB  SIZE  AND  SERIES 

2  =  DB  DIRECTORY 

3  =  DB  SYSTEM  DELETE 

4  •=  LIST  (MODIFY)  A  SYSTEM  HEADER 

5  =  INITIALIZE  A  DB  DISK 

6  =  MODIFY  A  SERIES  OF  SYSTEM  HEADERS 

7  -  POINT  TO  A  DATA  BASE 
0  =  EXIT  DBTILTY 
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The  ability  to  modify  a  data  base  due  to  problems  such  as  misspellings 
of  titles  or  improper  speaker  ID's  that  have  slipped  past  the  DRT 
scoring  program  is  possible  in  DBTILTY.  Yet,  the  package  does  not 
provide  any  means  of  modifying  the  data  itself.  This  feature  was 
purposely  not  provided.  It  is  our  philosophy  that  problems  with  the 
data  should  only  be  addressed  at  the  . ZFY  level.  Using  ZTILTY  allows 
booklet  and  page  modification  but  does  not  affect  the  scoring  of  a 
system.  Therefore  the  data  only  can  be  directly  modified  and  not  the 
scores.  Bad  data  resulting  from  wrong  wordlists  or  booklet  order  can 
be  deleted  using  DBTILTY  followed  by  the  correcting  of  the  original 
error  in  the  corresponding  NNNN.ZFY  file  with  ZTILTY  and  the  rescoring 
of  that  system  with  DRT. 

Most  of  the  options  have  been  described  in  previous  reports  (Ref.  7) . 
New  options  and  modifications  to  the  existing  options  are  presented  in 
the  next  section.  The  source  code  for  this  latest  version  of  DBTILTY 
is  available  from  RADC/EEV. 

DBTILTY  can  access  either  of  two  Analysis  DB  files  that  have  been 
mounted  on  pseudo  device  DK2 :  or  DK3:.  No  other  input  files  are  used 
with  the  exception  of  OPTION  6  which  will  be  discussed  in  a  later 
section.  The  only  output  file,  DBLIST.DMP,  is  created  by  OPTION  2  of 
DBTILTY  when  the  analyst  chooses  to  output  the  DB  volume  directory  to 
the  line  printer  (LP) . 

It  is  important  to  note  two  facts.  First,  this  package  can  effect 
important  data  files  and  should  only  be  used  by  personnel  that  fully 
understand  the  data  and  file  structures  involved.  Secondly,  before  any 
change  is  made  to  the  data  base  disk  the  program  will  execute  a  PAUSE 
to  give  the  user  a  final  chance  to  exit  or  abort  if  he  is  not  sure  of 
the  change . 


5.2.1  Option  Modifications 

The  DELETE  option  was  modified  to  handle  the  deleting  of  files  more 
efficiently.  The  previous  delete  option  would  find  the  system's 
location  in  the  INDEX  or  lOVFLO  arrays  and  replace  that  system  number 
with  zeroes  thus  causing  what  has  been  labeled  "holes"  in  the  Data 
Base.  "Holes"  in  the  lOVFLO  array  have  been  found  to  create  a  problem 
when  adding  new  versions  of  systems  that  already  exist.  If  a  "hole" 
existed  in  lOVFLO,  the  newer  version  would  be  added  before  the  older 
version  and  therefore  creating  an  out  of  sequence  version.  For 
example,  if  the  user  requested  a  listing  of  that  Data  Base,  the  newer 
version  just  added  would  appear  before  the  older  version  and  the  dates 
will  not  be  oldest  to  newest  as  they  should  be  for  analysis  purposes. 

To  eliminate  "holes",  the  DELETE  routine  moves  the  data  stored  in  the 
next  location  into  the  location  of  the  deleted  system  and  repeats  this 
similar  process  until  all  the  data  is  moved  up  therefore  fixing  the 
problem  of  a  hole  in  the  data  base.  The  revised  DELETE  option  will 
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also  work  on  expanded  probe  disks  in  a  similar  manner.  The  differences 
are  that  the  system  numbers  change  when  data  is  being  moved  up  since 
they  are  not  significant  to  the  system  and  the  INDEX  array  as  well  as 
the  lOVFLO  array  is  moved  up  to  eliminate  holes. 

The  previous  delete  option  only  allowed  the  user  to  delete  either  a 
specific  version  greater  than  1  or  all  versions  of  a  particular  system 
number.  The  revised  DELETE  allows  the  user  to  delete  any  system  and 
version  requested.  The  user  has  the  choice  of  deleting  all  versions  of 
a  particular  system  when  he  enters  the  system  number  and  version  1 
where  as  before  there  was  no  choice,  it  would  automatically  delete  all 
versions  even  if  the  user  just  wanted  to  delete  only  version  1. 

THE  DBLIST  option  was  revised  to  handle  the  listing  of  interrupted 
systems.  The  user  can  look  at  the  system  number  in  the  listing  of  the 
data  base  directory  and  see  what  systems  were  interrupted. 

5.2.2  New  Options 

A  new  service  option  was  implemented  into  the  DBTILTY  program  to 
enable  a  user  to  point  to  either  psuedo  device  DK2 :  or  DK3 ; . 
Previously,  the  assignment  had  to  be  made  each  time  an  option  was 
selected.  To  alleviate  this  inconvenience  the  POINT  TO  A  DATA  BASE 
option  was  added.  This  option  provides  an  easier  method  of  obtaining 
the  desired  Data  Base  simply  by  entering  the  DK  number  where  the  DB 
resides.  Then,  the  routine  assigns  the  appropriate  logical  unit 
numbers  to  the  three  files  of  the  Data  Base.  If  a  data  base  is  not 
found  at  the  DK  number  entered,  error  messages  appear  and  the  routine 
pauses  to  allow  the  user  to  assign  a  data  base.  The  user  must  enter  a 
RESUME  to  continue  executing  DBTILTY  once  the  assignment  has  taken 
place.  If  a  data  base  is  present,  the  routine  brings  up  the  main  menu 
once  again  and  execution  on  that  new  data  base  is  possible. 

Option  #6  was  implemented  into  DBTILTY  to  handle  modifying  a  series  of 
system  headers  rather  than  doing  each  one  individually  with  option  #4. 
The  new  option  uses  the  edited  output  file  LIST.DAT  created  from  the 
execution  of  the  GENDB  program.  The  file  LIST.DAT  must  be  renamed 
DBHEAD.TTL  before  executing  this  option. 

Option  #6  allows  the  user  to  do  the  following;  view  the  DBHEAD.TTL 
file  to  see  what  system  numbers  and  versions  are  contained  in  this 
file;  modify  all  versions  of  the  system  header  that  exist  for  each 
system  number  in  DBHEAD.TTL  without  user  intervention;  modify  each 
system  header  separately  with  user  intervention  (i.e.,  being  asked  to 
modify  this  system)  every  time  a  new  system  is  found;  or,  modify 
specific  system  numbers  and  versions  which  are  all  contained  in 
DBHEAD.TTL.  Due  to  the  implementation  of  the  modify  a  series  of 
headers  option,  the  problem  of  inconsistent  system  headers  within 
versions  of  the  same  system  can  be  resolved. 
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5.3  GENERIC  PROBE  DRT  DATA  BASE  GENERATION  PROGRAM 


A  program  was  developed  to  fill  a  GENERIC  PROBE  DRT  Data  Base  (GPRB) 
with  selected  systems  from  existing  Data  Bases.  The  program  is  called 
GENDB  and  can  be  found  on  the  DRTDEV  virtual  disk  at  [200,211].  The 
program  uses  Data  Bases  assigned  to  logicals  DK2 :  and  DK3 :  as  sources 
of  system  data  to  fill  the  Data  Base  assigned  the  logical  DB; .  The 
output  Data  Base,  namely  GPRB,  must  be  initialized  to  contain  system 
numbers  9600  to  9899  in  an  'extended  Probe'  format.  The  9600  series 
indicates  that  this  is  a  generic  probe  DB. 

The  selection  of  the  systems  from  the  source  Data  Bases  can  be  done  in 
one  of  three  ways : 

1.  Systems  to  be  selected  are  specified  in  order  within  a  text 
file  LIST.DAT  which  can  be  generated  with  EDT  or  by  previous 
runs  of  GENDB.  Each  record  of  LIST.DAT  contains  a  system 
number,  a  version  number,  and  the  title  field  from  the  header 
record  for  the  system.  (Note:  if  generating  LIST.DAT  with 
EDT,  only  the  system  and  version  numbers  are  required.) 

2.  Systems  to  be  selected  are  specified  interactively  with  the 
user  typing  in  system  and  version  number  during  the  execution 
of  GENDB. 

3.  Selection  of  systems  based  upon  specified  contents  of  the 
title  field  within  its  header  record.  The  title  field  can  be 
used  in  evaluating  expressions  consisting  of  substrings  and 
the  boolean  operators  OR,  AND,  and  NOT.  If  the  resulting 
expression  evaluation  returns  a  true  value,  the  system  is 
selected  for  inclusion  in  the  output  Data  Base.  An  order  of 
the  evaluation  of  systems  is  also  possible  by  using 
constructs  such  as  [ 'xxxx' .OR. 'yyyy ' ]  where  a  search  for  the 
'xxxx'  substring  is  carried  out  first  and  then  a  second 
search  is  made  for  the  other  substring.  More  than  one  of 
these  structures  can  be  given  within  a  search  expression  for 
multiple  layers  of  ordering.  The  option  to  require  the 
presence  of  specified  speakers  in  the  system  is  also 
available . 


Once  systems  have  been  selected  and  the  LIST.DAT  file  is  generated, 
the  user  has  the  option  of  reviewing  and  revising  this  list  by  using 
the  EDT  utility  which  is  spawned  by  GENDB  on  command.  The  edited 
LIST.DAT  is  used  during  the  data  transfer  process  in  which  system  data 
is  copied  into  the  GPRB  Data  Base.  Previous  data  in  GPRB  is 
overwritten  when  GENDB  is  executed. 

The  source  code  for  GENDB  is  available  from  RADC/EEV. 
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CHAPTER  6 


j 

DRT  TESTING  OF  INTERRUPTED  VOICE  COMMUNICATION  SYSTEMS 


New  capabilities  have  been  developed  at  the  RADC/EEV  Speech  Processing 
Facility  for  simulating  noisy,  delayed  digital  communication  channels 
and  for  obtaining  Diagnostic  Rhyme  Test  (DRT)  scores  for  communication 
systems  operating  over  the  simulated  channel.  Intelligibility  scores 
for  processed  speech  transmitted  over  the  channel  can  be  obtained  by 
using  a  DRT  test  procedure  which  generates  scores  (i.e.,  percentage  of 
words  correctly  recognized)  on  the  basis  of  only  test  items 
successfully  transmitted  while  items  "interrupted"  by  error  bursts  are 
disregarded  during  the  DRT  scoring  process . 

Testing  of  "interrupted  systems"  with  the  modified  DRT  does  not 
provide  a  complete  measure  of  the  intelligibility  of  the  system  by 
itself.  This  score  must  be  combined  with  a  measure  of  the  percent  of 
transmission  time  that  the  communications  channel  has  zero 
intelligibility  (i.e.  the  channel  is  effectively  shut  down).  The 
Channel  Down  Time  (CDT)  measure  plus  the  modified  DRT  score  gives  a 
complete  picture  of  the  system's  intelligibility.  CDT  measures  will  be 
discussed  in  this  Chapter. 

The  software  modifications  to  the  RADC/EEV  DRT  System  for  the 
incorporation  of  interrupted  speech  communications  intelligibility 
testing  have  been  covered  in  Chapters  2  -  5  of  this  report.  A 
procedure  for  the  selection  of  which  test  words  are  missing  from  a  DRT 
tape  is  discussed  in  Chapter  3  of  this  report.  This  Chapter  will  give 
the  background  in  the  development  of  the  test  method,  a  justification 
for  the  method  and  results  that  validate  the  approach. 


6.1  A  STUDY  OF  THE  D^  ^  A  MEASURE  OF  INTELLIGIBILITY  WITH 
INTERRUPTED  VOICE  COMMUNICATION  SYSTEMS 

Intelligibility  testing  of  some  voice  communication  systems  can 
include  the  situation  where  silence  or  garbled  noise  replaces  the 
speech  signal  at  the  receiver  for  periods  of  time  from  1  to  10 
seconds.  Often  a  squelch  algorithm  has  been  used  to  insert  silence 
when  high  transmission  errors  are  detected.  Such  systems  will  be 
referred  to  as  "interrupted  systems"  in  this  report. 
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The  standard  tool  for  the  measurement  of  intelligibility  of  voice 
communication  systems  is  the  Diagnostic  Rhyme  Test  (DRT) .  In  the  DRT 
individual  words  from  rhyming  pairs  are  presented  by  audio  tape  to  a 
listener  crew  at  a  pace  of  approximately  1.3  seconds.  At  ElADC/EEV  a 
listener  responds  to  each  word  presented  by  striking  a  key  on  a  data 
entry  unit  (DEU)  to  indicate  which  word  the  listener  believes  he/she 
heard.  This  response  scrolls  a  word-pair  display  on  the  screen  of  the 
DEU.  The  response  keeps  the  word-pair  display  in  synchronization  with 
the  audio  tape  word  presentation.  If  a  listener  does  not  respond  to  a 
word  choice,  the  DEU  display  will  fall  behind  the  audio  tape  and  the 
listener  can  lose  place  in  the  test. 

For  interrupted  systems  the  location  of  silent  or  garbled  periods 
tends  to  be  scattered  randomly  within  a  DRT  tape .  This  results  in 
"missing  words"  within  the  word  list.  Since  these  missing  words  are  on 
the  audio  tape,  they  will  require  all  members  of  the  listener  crew  to 
guess  at  what  word  should  have  been  presented.  A  guessing  correction 
algorithm  is  used  in  the  scoring  procedure  for  the  DRT.  This  algorithm 
weights  wrong  answers  by  2  and  correct  answers  by  1 .  A  50/50  chance 
test  with  all  guesses  will  result  in  a  score  of  0.0,  a  clear  system 
with  all  correct  answers  will  score  100.0,  and  a  system  that  has  all 
wrong  answers  will  score  a  -100.0. 

Testing  of  "interrupted  systems"  with  the  DRT  at  RADC/EEV  does  not 
provide  a  complete  or  accurate  measure  of  the  intelligibility  of  the 
system.  Since  the  missing  words  are  scattered  throughout  the 
wordlists,  the  diagnostic  attribute  or  feature  scores  provided  by  the 
DRT  become  inaccurate.  An  adjusted  score  can  be  provided  if  the 
specific  missing  words  are  known  for  a  given  word  list.  An  adjusted 
DRT  report  coupled  with  a  measure  of  the  percent  time  the  transmission 
was  affected  by  squelch  or  garble  will  provide  a  complete  measure  of 
the  system's  intelligibility. 

The  most  important  aspect  of  an  interrupted  DRT  is  still  not  addressed 
by  the  missing  word  correction.  Do  missing  word  sequences  effect 
future  word  choices  because  of  synchronization  loss  by  the  listener 
crew?  Do  the  listeners  lose  their  place  between  the  audio  tape  and  DEU 
word  display  and  if  so,  how  long  does  it  take  a  trained  listener  to 
catch  up?  This  type  of  error  will  be  referred  to  as  Follow  Through  or 
Projected  Error.  A  controlled  experiment  has  been  conducted  to  answer 
these  questions. 


6.1.1  Experimental  Design 

A  3-Speaker  DRT  wordlist  (JE,CH,RH  3Male  Quiet/Dynamic  Mic)  available 
on  the  EEV  Speech  Data  Base  (DB)  was  modified  with  the  ILS  package. 
Words  were  removed  from  the  list  and  replaced  with  silence,  acoustic 
noise  or  transmission  noise.  All  testing  was  done  with  96kbps  PCM 
speech  as  output  from  the  Speech  DB  by  MAPOUT.  Up  to  five  consecutive 
words  were  replaced,  resulting  in  interruptions  of  from  1  to  6.5 
seconds.  The  DRT  wordlists  are  structured  with  sets  of  six  test  words, 
for  the  six  attributes  being  evaluated,  separated  by  spacer  or 
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experimental  words.  Since  the  DRT  was  originally  setup  to  be  performed 
with  paper  booklets,  there  are  also  spacer  words  at  the  tops  and 
bottoms  of  columns  of  word  choices.  There  are  four  pages  within  a  DRT 
booklet  and  extra  time  is  given  within  the  audio  tapes  for  page  turns . 
This  structure  allows  for  the  removal  of  up  to  two  consecutive  words 
and  the  introduction  of  several  seconds  of  noise  without  effecting  the 
intelligibility  measurement  of  a  system. 

Two  sets  of  tests  (A  &  B)  were  generated  from  the  Speech  DB  file  of 
DRT  tape  D-2-A.  Each  set  has  three  test  tapes,  one  each  for  words 
replaced  with  silence,  acoustic  noise,  and  transmission  noise.  EC- 130 
aircraft  acoustic  background  noise  was  used  to  replace  the  words  in 
the  acoustic  noise  version  of  the  test.  A  simulated  transmission 
channel  noise,  taken  from  DRT  tapes  with  actual  burst  error 
conditions,  was  used  to  replace  the  words  in  the  transmission  channel 
noise  version  of  the  test. 

Set  A  has  33  replaced  words  in  total  with  11  of  these  being  test 
words.  Set  B  is  an  extension  of  Set  A  with  a  total  of  45  replaced 
words  with  23  of  these  being  test  words.  The  effects  on  DRT  results 
from  test  words  that  are  removed  can  be  calculated  for  the  case  where 
no  listener  synchronization  loss  has  taken  place.  Tables  6.1  St  6.2 
give  details  on  the  numbers  and  types  of  words  removed  along  with  the 
predicted  effects  for  the  DRT  results  as  given  after  the  "/".  The 
specific  words  removed  and  the  sequences  of  words  involved  are  given 
in  the  Figure  6.1  in  the  form  of  a  DRT  Booklet. 


Table  6.1  SET  A 

Replaced  Word  Attributes  and  Predicted  DRT  Effects 
System  2669  Silence  Replaces  Words 
System  2670  Acoustic  Noise  Replaces  Words 
System  2671  Transmission  Burst  Error  Noise  Replaces  Words 


# 

#Spacer 

#Test 

# 

# 

# 

# 

# 

# 

Words 

Words 

Words 

V 

N 

Su 

Si 

G 

C 

Spkr  JE  11 

9 

2/1.0 

- 

- 

- 

- 

- 

2/6.2 

Spkr  CH  11 

7 

4/2.1 

4/12.5 

- 

- 

- 

- 

- 

Spkr  RH  11 

6 

5/2.6 

2/6.2 

- 

- 

- 

- 

3/9.4 

Total  33 

22 

11/1.9 

6/6.2 

- 

- 

- 

- 

5/5.2 

Table  6.2  SET  B 

Replaced  Word  Attributes  and  Predicted  DRT  Effects 
System  2672  Silence  Replaces  Words 
System  2673  Acoustic  Noise  Replaces  Words 
System  2674  Transmission  Burst  Error  Noise  Replaces  Words 


# 

Words 

Spkr  JE  16 
Spkr  CH  16 
Spkr  RH  13 
Total  45 


#Spacer  #Test 


Words  Words 

9  7/3.6 

7  9/4.7 

6  7/3.6 

22  23/3.9 


#  #  #  #  #  # 

V  N  Su  Si  G  C 

2/6.2  -  -  1/3.1  1/3.1  3/9.4 

5/15.6  1/3.1  1/3.1  -  1/3.1  1/3.1 

2/6.2  -  -  1/3.1  1/3.1  3/9.4 

9/9.4  1/1.0  1/1.0  2/2.1  3/3.1  7/7.3 
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DIAGNOSTIC  RHYME  TEST  DIAGNOSTIC  RHYME  TEST 
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Figure  6.1  DRT  Booklet  (  Pages  1  and 


DIAGNOSTIC  RHYME  TEST  DIAGNOSTIC  RHYME  TEST 
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Figure  6.1  (Continued)  (  Pages  3  and 


6.1.2  Data  Analysis 


The  modified  wordlist  tapes  were  evaluated  by  the  EEV  listener  crew, 
with  several  repeats  carried  out  for  each  tape.  Test  results  from  a 
base  line  system  evaluation  (no  missing  words)  are  provided  in  Table 
6.3.  These  results  have  been  averaged  over  repeats  of  the  system. 
Averaged  test  results  for  the  systems  with  missing  or  replaced  words 
are  given  in  Tables  6.4  -  6.9.  These  tables  also  give  the  actual  and 
predicted  differences  (delta  =  baseline  -  interrupted) .  The  effect 
that  we  are  trying  to  isolate  is  a  Projected  Error  on  words  following 
a  sequence  of  replaced  or  missing  words.  The  tables  indicate  chose 
attribute  scores  where  this  effect  will  be  seen  if  it  exists.  The 
number  of  times  (n)  that  an  attribute  word  follows  a  number  of 
replaced  words  (m)  is  indicated  as  n*Dm.  There  could  also  be  a  cascade 
effect  that  would  be  noticed  in  the  score  of  the  second  attribute 
following  a  missing  sequence.  It  should  be  noted  that  these  are  not 
full  bandwidth  analog  systems,  but  bandlimited  (lOOHz  HP  /  4000Hz  LP) 
96kbps  PCM  systems.  Also  note  that  the  standard  errors  (S.E.)  given 
are  across  listener  by  speaker  interactions  as  averaged  across  systems 
for  all  of  these  tables. 


Table  6.3 


SYSTEM  2656 

PDP- 

11/44  D-2 

-A  DISK 

MASTER 

3M/Q/DYN 

FEATURES 

V 

N 

SU 

SI 

G 

C 

TOTAL 

SPEAKER  NUMBER 

1 

P-^A:  MEAN 

86.2 

99.5 

93.1 

99.1 

94.8 

99.2 

95.3 

S.E. 

0.9 

0.2 

1.5 

0.5 

0.9 

0.6 

0.3 

SPEAKER  NUMBER 

2 

P+A:  MEAN 

98.8 

99.5 

98.8 

98.8 

91.2 

98.9 

97.7 

S.E. 

0.4 

0.3 

0.5 

0.7 

1.2 

0.6 

0.2 

SPEAKER  NUMBER 

3 

P-i-A:  MEAN 

98.3 

98.6 

99.8 

97.2 

93.4 

98.6 

97.7 

S.E. 

0.6 

0.8 

0.2 

0.8 

0.3 

0.9 

0.2 

SPEAKER  AVERAGE 

P-t-A;  MEAN 

94.4 

99.2 

97.2 

98.3 

93.2 

98.9 

96.9 

S.E. 

1.3 

0.3 

0.8 

0.4 

0.6 

0.4 

0.3 

THIS  REPORT  IS  THE  AVERAGE  OF  THE  FOLLOWING  SYSTEMS 

SYSTEM  2656  VER  1  -  5  (20-NOV-85 ; 22-NOV- 85 ; 29 -NOV- 85 ; 09 - JUL- 86 ; 11 - JUL- 86 ) 
THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 


6-6 


Table  6.4  SET  A 

SYSTEM  2669  INTERRUPTED  DRT  ZEROED  WORDS  D2A  3M/Q/DYN 

FEATURES 


SPEAKER  NUMBER 

V 

1 

N 

SU 

SI 

G 

C 

TOTAL 

P+A:  MEAN 

86.1 

99.0 

93.5 

98.8 

94.7 

92.3 

94.0 

S.E. 

1.2 

0.3 

1.1 

0.4 

0.7 

0.7 

0.4 

DELTA  ACT/PRED 

SPEAKER  NUMBER 

.2/0. 

3*D2 

2 

.6/0. 

- .4/0. 

.2/0. 

.2/0. 

6. 9/6. 2 

1. 3/1.0 

P+A:  MEAN 

84.9 

98.8 

97.9 

98.2 

91.4 

98.8 

95.0 

S.E. 

0.8 

0.2 

0.7 

0.6 

1.5 

0.4 

0.2 

DELTA  ACT/PRED 

SPEAKER  NUMBER 

13.8/12.5 

3 

.7/0. 

4V.D2 

.8/0. 

.6/0. 

-  .2/0. 

.1/0. 

2. 6/1. 9 

P+A;  MEAN 

93.4 

97.1 

99.1 

96.4 

94.7 

90.9 

95.2 

S.E. 

0.9 

1.0 

0.2 

1.1 

1.5 

0.5 

0.2 

DELTA  ACT/PRED  4 . 9/6 . 2 
1*D2/1*D3 
SPEAKER  AVERAGE 

1.4/0. 

2+D3 

.8/0. 

.8/0. 

-1.2/0. 

7. 7/9.4 

2. 4/2. 6 

P+A:  MEAN 

88.1 

98.3 

96.8 

97.8 

93.6 

94.0 

94.8 

S.E. 

1.0 

0.4 

0.7 

0.5 

0.8 

0.8 

0.2 

DELTA  ACT/PRED 

6. 3/6. 2 

.9/0. 

.4/0. 

.5/0. 

-  .4/0. 

4. 9/5. 2 

2. 1/1. 9 

THIS  REPORT  IS  THE  AVERAGE  OF  THE  FOLLOWING  SYSTEMS 

SYSTEM  2669  VER  1  -  6  (25-JUN-86  A:25-JUN-86  B;26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 

Table  6.5  SET  A 

SYSTEM  2670  INTERRUPTED  DRT  EC- 130  NOISEWORDS  D2A  3M/Q/DYN 

FEATURES 


SPEAKER  NUMBER 

V 

1 

N 

SU 

SI 

G 

C 

TOTAL 

P+A;  MEAN 

86.7 

99.1 

95.2 

99.4 

95.6 

93.1 

94.8 

S.E. 

2.0 

0.4 

1.1 

0.4 

0.8 

0.7 

0.5 

DELTA  ACT/PRED 

SPEAKER  NUMBER 

-.5/0. 

3*D2 

2 

.4/0. 

-2.0/0. 

-.3/0. 

-.7/0. 

6. 1/6. 2 

.5/1.0 

P+A:  MEAN 

85.7 

98.8 

98.2 

97.7 

91.9 

99.2 

95.2 

S.E. 

1.1 

0.6 

0.7 

0.8 

1.2 

0.4 

0.3 

DELTA  ACT/PRED 

SPEAKER  NUMBER 

13.0/12. 

3 

5  .7/0. 
4*D2 

.6/0. 

1.1/0. 

-.7/0. 

-  .3/0. 

2. 4/1. 9 

P+A:  MEAN 

94.1 

97.4 

99.6 

96.5 

93.9 

91.8 

95.6 

S.E. 

1.0 

0.6 

0.2 

1.0 

0.5 

1.0 

0.4 

DELTA  ACT/PRED  4. 1/6. 2 
1*D2/1*D3 
SPEAKER  AVERAGE 

1.2/0. 

2*D3 

.2/0. 

.7/0. 

-  .4/0. 

6. 8/9. 4 

2. 1/2. 6 

P+A:  MEAN 

88.8 

98.4 

97.7 

97.8 

93.8 

94.7 

95.2 

S.E. 

1.1 

0.3 

0.6 

0.5 

0.6 

0.8 

0.2 

DELTA  ACT/PRED 

5. 6/6. 2 

.8/0. 

- .4/0. 

5/0. 

-  . 6/0 . 

4. 2/5. 2 

1. 7/1.9 

THIS  REPORT 

IS  THE  , 

AVERAGE 

OF  THE 

FOLLOWING 

SYSTEMS 

SYSTEM  2670  VER  1  -  6  (25-JUN-86  A;25-JUN-86  B:26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 
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Table  6.6  SET  A 

SYSTEM  2671  INTERRUPTED  DRT  BURST  WORDS  D2A  3M/Q/DYN 

FEATURES 


SPEAKER  NUMBER 

V 

1 

N 

SU 

SI 

G 

C 

TOTAL 

P-t-A;  MEAN 

87.2 

98.7 

95.0 

99.1 

95.3 

92.8 

94.7 

S.E. 

1.8 

0.7 

1.1 

0.4 

1.2 

0.9 

0.4 

DELTA  ACT/PRED 

SPEAKER  NUMBER 

-1.0/0. 

3*D2 

2 

.8/0. 

-1.9/0. 

O./O. 

-.5/0. 

6.4/6. 2 

.6/1.0 

P+A;  MEAN 

85.6 

99.6 

98.0 

97.4 

91.2 

98.7 

95.1 

S.E. 

1.0 

0.2 

0.6 

0.5 

1.0 

0.6 

0.3 

DELTA  ACT/PRED 

SPEAKER  NUMBER 

13.2/12.5 

3 

-.1/0. 

4*D2 

.7/0. 

1.4/0. 

.1/0. 

.2/0. 

2. 6/1. 9 

P+A:  MEAN 

92.2 

96.7 

99.7 

97.0 

94.1 

90.5 

95.0 

S.E. 

1.3 

0.9 

0.2 

1.1 

1.4 

0.5 

0.3 

DELTA  ACT/PRED  6 . 1/6 . 2 
1*D2/1*D3 

SPEAKER  AVERAGE 

1.8/0. 

2*D3 

.1/0. 

.2/0. 

-.7/0. 

8. 1/9. 4 

2. 6/2. 6 

P+A:  MEAN 

88.3 

98.4 

97.6 

97.8 

93.5 

94.0 

94.9 

S.E. 

1.0 

0.4 

0.6 

0.4 

0.7 

0.8 

0.2 

DELTA  ACT/PRED 

6. 1/6. 2 

.9/0. 

- .4/0. 

.5/0. 

- . 4/0 . 

4. 9/5. 2 

1.9/1. 9 

THIS  REPORT  IS  THE  AVERAGE  OF  THE  FOLLOWING  SYSTEMS 

SYSTEM  2671  VER  1  -  6  (25-JUN-86  A;25-JUN-86  B;26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 

Table  6.7  SET  B 

SYSTEM  2672  INTERRUPTED  DRT  ZEROED  WORDS  D2A  3M/Q/DYN 

FEATURES 


V 

N 

SU 

SI 

G 

C 

TOTAL 

SPEAKER  NUMBER 

1 

P+A:  MEAN 

80.7 

98.7 

92.4 

95.6 

95.6 

90.9 

92.3 

S.E. 

1.9 

0.6 

1.5 

1.1 

0.6 

1.0 

0.4 

DELTA  ACT/PRED 

5.5/6. 

2  .8/0. 

.7/0. 

3. 5/3.1 

-.7/3.1 

8. 3/9.4 

3. 0/3. 6 

2*D2 

1*D4/1*D5 

SPEAKER  NUMBER 

2 

P+A:  MEAN 

81.8 

94.0 

93.5 

97.9 

91.7 

98.2 

92.8 

S.E. 

1.2 

1.1 

0.6 

0.7 

1.7 

0.5 

0.3 

DELTA  ACT/PRED 

17.0/15 

.6  5. 5/3.1 

5.3/3. 

1  .8/0. 

-  .4/3.1 

.7/3.1 

4. 8/4. 7 

3*D2/1*D4 

1*D4 

SPEAKER  NUMBER 

3 

P+A:  MEAN 

93.2 

98.2 

99.2 

94.5 

90.1 

92.2 

94.6 

S.E. 

1.8 

0.7 

0.4 

1.7 

1.6 

1.1 

0.8 

DELTA  ACT/PRED 

5.0/6. 

2  .4/0. 

.6/0. 

2. 7/3.1 

3. 3/3.1 

6. 4/9. 4 

3. 1/3. 6 

1*D2/1*D5  2*D3 

SPEAKER  AVERAGE 

P+A:  MEAN 

85.2 

97.0 

95.0 

96.0 

92.4 

93.8 

93.2 

S.E. 

1.5 

0.6 

0.8 

0.7 

0.9 

0.8 

0.4 

DELTA  ACT/PRED 

9.2/9. 

4  2. 3/1.0  2 

.2/1.0 

2 . 3/2 . 1 

.7/3.1 

5.2/7. 3 

3. 6/3. 9 

THIS  REPORT 

IS  THE 

AVERAGE  OF 

THE  FOLLOWING 

SYSTEMS 

SYSTEM  2672 

VER  1 

-  3  (09-JUL 

-86;11 

-JUL-86  A;11-JUL- 

86  B) 

THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 
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Table  6.8  SET  B 

SYSTEM  2673  INTERRUPTED  DRT  EC- 130  NOISEWORDS  D2A  3M/Q/DYN 

FEATURES 


V 

N 

SU 

SI  G 

C 

TOTAL 

SPEAKER  NUMBER 

1 

P+A:  MEAN 

81.5 

97.9 

94.3 

95.8  93.0 

90.9 

92.2 

S.E. 

1.8 

0.8 

1.4 

1.0  1.0 

0.9 

0.5 

DELTA  ACT/PRED 

4.7/6. 

2  1.6/0.  -1 

.1/0. 

3.2/3. 1  1. 9/3.1 

8. 3/9.4 

3. 1/3.6 

2*D2 

1*D4/1*D5 

SPEAKER  NUMBER 

2 

P+A;  MEAN 

80.2 

96.1 

91.4 

97.4  89.8 

96.6 

91.9 

S.E. 

2.0 

0.9 

1.3 

0.6  1.6 

0.7 

0.4 

DELTA  ACT/PRED 

18.5/15 

.6  3.4/3. 1 

7.3/3 

.1  1.4/0.  1.4/3. 1 

2. 3/3.1 

5. 7/4. 7 

3*D2/1*D4 

1*D4 

SPEAKER  NUMBER 

3 

P+A;  MEAN 

90.1 

96.9 

98.7 

93.5  91.7 

92.4 

93.9 

S.E. 

2.3 

0.9 

0.6 

1.1  1.2 

1.4 

0.6 

DELTA  ACT/PRED 

8.2/6. 

2  1.7/0.  1 

.1/0. 

3. 7/3.1  1. 8/3.1 

6. 1/9.4 

3. 8/3. 6 

1*D2/1*D5  2*D3 

SPEAKER  AVERAGE 

P+A;  MEAN 

83.9 

97.0 

94.8 

95.6  91.5 

93.3 

92.7 

S.E. 

1.5 

0.5 

0.9 

0.6  0.8 

0.8 

0.3 

DELTA  ACT/PRED 

10.5/9. 

4  2. 3/1.0  2 

.4/1. 

0  2. 8/2.1  1. 7/3.1 

5. 6/7. 3 

4. 2/3. 9 

THIS  REPORT 

IS  THE 

AVERAGE  OF 

THE 

FOLLOWING  SYSTEMS 

SYSTEM  2673 

VER  1 

-  3  (09-JUL 

-86  A;09-JUL-86  B;ll-JUL-86) 

THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 

Table  6.9  SET  B 

SYSTEM  26 7A  INTERRUPTED  DRT  BURST  WORDS  D2A  3M/Q/DYN 

FEATURES 


V 

N 

SU 

SI 

G 

C 

TOTAL 

SPEAKER  NUMBER 

1 

P+A;  MEAN 

83.1 

98.7 

96.1 

96.4 

92.4 

90.1 

92.8 

S.E. 

2.1 

0.9 

1.4 

1.1 

1.1 

0.9 

C.6 

DELTA  ACT/PRED 

3.2/6. 

2  .8/0.  -3 

.0/0. 

2. 7/3.1 

2.4/3. 1 

9. 1/9. 4 

2. 5/3. 6 

2*D2 

1*D4/1*D5 

SPEAKER  NUMBER 

2 

P+A:  MEAN 

83.6 

96.4 

94.5 

98.7 

90.9 

95.3 

93.2 

S.E. 

1.2 

0.5 

0.9 

1.0 

1.7 

1.2 

0.3 

DELTA  ACT/PRED 

15.2/15 

.6  3. 2/3.1 

4.2/3 

.1  O./O. 

.4/3.1 

3.6/3. 1 

4 . 4/4 . 7 

3*D2/1*D4 

1*D4 

SPEAKER  NUMBER 

3 

P+A;  MEAN 

92.2 

97.1 

99.0 

93.0 

91.9 

92.4 

94.3 

S.E. 

0.9 

1.2 

0.7 

1.5 

1.5 

1.5 

0.7 

DELTA  ACT/PRED 

6.1/6. 

2  1.4/0. 

.9/0. 

4. 2/3.1 

1. 5/3.1 

6. 1/9.4 

3. 4/3. 6 

1*D2/1*D5  2*D3 

SPEAKER  AVERAGE 

P+A:  MEAN 

86.3 

97.4 

96.5 

96.0 

91.8 

92.6 

93.4 

S.E. 

1.2 

0.6 

0.7 

0.8 

0.8 

0.8 

0.3 

DELTA  ACT/PRED 

8.2/9. 

4  1. 8/1.0 

.7/1. 

0  2. 3/2.1 

1.4/3. 1 

6. 3/7. 3 

3. 4/3. 9 

THIS  REPORT 

IS  THE 

AVERAGE  OF 

'  THE 

FOLLOWING 

SYSTEMS 

SYSTEM  2674  VER  1  -  3  (09-JUL-86  A;09-JUL-86  B:ll-JUL-86) 

THIS  REPORT  IS  FOR  8  LISTENERS  PER  SYSTEM  -  SPEAKERS  JE  CH  RH 
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For  now  we  will  limit  our  analysis  of  the  data  to  verification  of  the 
existence  of  the  Projected  or  Follow  Through  Error  effect.  For  Set  A 
we  do  not  find  depressed  attribute  scores  for  the  various  cases  of  two 
missing  words  (D2).  For  Speaker  RH  there  is  no  effect  found  with  three 
missing  words  (D3)  following  the  attribute  Voicing,  but  there  is  a 
possible  effect  for  the  attribute  Nasality  with  two  three  missing  word 
sequences  (2*D3)  where  we  have  a  score  that  is  depressed  by  1  DRT 
point.  This  corresponds  to  17%  of  the  listener  crew  losing  synch  for 
one  word  each  time.  The  effect  is  not  evident  for  Speaker  RH's  total 
score  or  for  the  Nasality  score  averaged  across  the  three  Speakers. 
These  results  are  consistent  across  all  three  word  replacement  states 
of  Set  A  (silence,  acoustic  noise,  burst  channel  noise).  There  is  no 
evidence  of  a  cascade  effect  past  the  first  word  following  a  missing 
sequence.  The  Voicing  score  for  Speaker  RH  preceded  by  a  3  missing 
word  case  is  also  affected  by  having  two  Voicing  words  replaced.  This 
score  is  not  only  not  depressed,  but  is  greater  than  expected  by  2  DRT 
points.  If  the  Projected  Error  effect  is  present,  it  is  very  small  for 
missing  sequences  up  to  3  words. 

Set  B  includes  sequences  of  up  to  5  missing  words.  The  same  small 
effect  seen  with  Set  A  is  evident  for  Speaker  JE,  acoustic  noise, 
attribute  Nasality  with  sequences  of  4  and  5  missing  words  and  for 
Speaker  RH,  acoustic  noise,  attribute  Nasality  with  two  sequences  of  3 
missing  words.  The  effect  on  Speaker  RH  could  also  contain  a  cascade 
from  the  5  missing  word  state  on  Voicing.  There  may  also  be  a  cascade 
effect  evident  for  RH,  acoustic  noise,  attribute  Sustension.  All  of 
these  effects  are  of  the  same  size  as  that  found  for  Set  A  (i.e.  a  17% 
listener  crew  synch  loss).  Similar  effects  are  noted  for  Speaker  CH, 
silence,  attribute  Nasality  and  acoustic  noise,  attribute  Sibilance. 
There  is  a  major  cascade  effect  across  all  word  replacement  states  for 
Speaker  CH,  attribute  Sustension.  The  effect  is  equivalent  to  all 
listeners  missing  one  extra  Sustension  test  word  during  four  possible 
cascade  situations.  The  attribute  Nasality  that  should  also  have  been 
affected  if  this  was  a  cascade  effect  is  only  depressed  for  the 
silence  state.  A  closer  look  at  the  variability  of  the  listener 
sustension  scores  in  the  System  repeats  that  were  averaged,  indicates 
that  the  95%  C.I.  for  this  score  is  as  large  as  the  measured 
deviation.  This  indicates  that  the  cascade  effect  is  as  likely  not 
present  as  it  is  present. 

Elsewhere  in  the  results  for  Set  B  there  are  incidences  of  inflated 
scores  especially  for  the  attribute  Graveness  across  all  word 
replacement  states  and  Speakers.  A  possible  Left/Right  word  choice  vs 
word  removed  bias  was  checked  as  a  source  of  this  anomaly.  This  bias 
was  not  found  to  be  present.  A  reason  for  these  inflated  scores  has 
not  been  found.  Effects  between  the  silence,  acoustic  noise,  and  block 
error  noise  word  replacement  states  were  expected.  It  was  thought  that 
the  presence  of  any  signal  rather  than  silence  would  help  the 
listeners  maintain  synchronization.  Since  the  listeners  appeared  to 
have  kept  synch  for  the  silence  state,  no  positive  effect  for  the 
noise  states  was  found.  There  may  have  been  an  "irritation"  effect 
present  with  the  noise  states  for  the  longer  replaced  word  sequences 
found  in  Set  B. 
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An  extensive  Analysis  of  Variance  (ANOVA)  across  the  individual  System 
tests  with  comparisons  to  the  individual  base  line  System  tests  shows 
significant  differences  at  a  95%  Confidence  Interval  (C.I.)  for  only 
those  attributes  that  have  been  modified  (i.e.  V  &  C  for  Set-A;  All 
for  Set-B) .  This  analysis  was  for  attribute  scores  averaged  across 
speakers,  however,  a  significant  difference  was  also  indicated  for 
System  by  Speaker  by  Attribute  interactions.  No  significant  effect  was 
indicated  between  the  methods  of  word  replacement  (i.e.  silence, 
acoustic  noise,  burst  noise). 


6.1.3  Study  Results 

The  results  of  this  study  indicate  that  "interrupted  DRT's"  can  be 
carried  out  by  trained  listener  crews  without  worry  about 
synchronization  or  guessing  problems.  Any  method  that  provides  the 
listener  crew  with  an  indication  that  a  word  is  expected  (i.e.  forced 
pacing,  visual  or  audible  indicators)  would  require  retraining  of  the 
crew  and  limit  any  comparisons  between  past  ^nd  future  testing.  It 
would  also  further  differentiate  the  RADC  DRT  facility  from  other  DRT 
facilities  and  would  require  changes  in  the  test  tape  preparation 
method.  The  introduction  of  a  "guess"  or  "missing  word"  key  to  the 
DEU's  would  require  extensive  retraining  of  the  listener  crew  and 
could  cause  listeners  to  use  this  key  rather  that  make  the  fine  word 
pair  distinctions  that  are  now  required  of  them.  The  storage  and  data 
transfer  of  a  tri-state  variable  (i.e.  right/left/missing  word)  would 
require  a  complete  modification  of  all  DRT  System  software,  file 
structures  and  the  DRT  DB. 


6.2  D^  MODIFICATIONS 

The  adaptation  of  the  DRT  that  is  required  is  an  adjusted  scoring 
method  that  takes  into  account  the  "missing"  words.  These  "missing" 
words  will  have  to  be  identified  by  some  process,  and  decisions  about 
words  that  are  only  partially  affected  will  have  to  be  made.  With  the 
words  identified,  a  modification  to  the  scoring  algorithm  can  be  made 
rather  simply.  Since  each  DRT  of  this  type  will  have  different 
"missing"  word^  the  procedure  to  locate  them  will  have  to  be  simple 
to  carry  out,  yet  highly  accurate.  A  time  base  file  for  the  start  of 
each  word  in  the  working  master  could  be  used  as  a  mask  with  the 
processed  tape  to  flag  the  "missing"  words.  An  analog  tape  of  the  TTL 
output  levels  of  the  BER  program  could  also  be  used  as  a  mask.  ILS 
could  then  be  used  for  verification.  This  process  would  require  that 
the  time  mask  file  and  the  test  tape  be  input  to  the  RADC  Speech  DB. 
The  DRT  DB  would  have  to  be  modified  to  maintain  the  precision 
currently  available.  This  is  because  the  DRT  DB  stores 
Speaker-Listener  interactions  summed  over  attribute  test  tokens.  The 
range  used  is  +16  to  -16  and  a  constant  100. /16.  is  used  to  convert 
scores  to  percent.  With  a  base  number  of  tokens  less  than  16,  there 
are  quantization  levels  that  can  not  be  represented. 
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The  DRT  software  and  Data  Base  has  been  modified  to  handle 
"Interrupted  DRT  Systems."  A  new  version  of  the  scoring  routine  DRT 
handles  the  listener  data  with  the  help  of  a  KEY  file  that  defines  the 
missing  words.  This  routine  provides  adjusted  results  in  report  form 
and  transfers  the  test  data  to  the  DRT  Data  Base.  Details  of  the  new 
DRT  program  are  covered  in  Chapter  Three  of  this  report.  A 
modification  of  the  DB  structure  allows  for  the  storage  of 
"interrupted  DRT  Systems"  within  the  existing  DB  without  effecting  the 
storage  or  analysis  of  normal  DRT  system  results.  Details  of  the  DB 
modifications  are  covered  in  Chapter  Two  of  this  report.  A  new  version 
of  the  MENU  package  had  to  be  developed  to  provide  comparative  results 
and  analysis  between  normal  and  interrupted  systems.  The  changes  to 
the  comparative  analysis  package  MENU  are  covered  in  Chapter  Four  of 
this  report. 


6.2.1  Results  With  Modified  DRT 


The  new  software  was  used  to  score  and  analyze  the  interrupted  DRT 
Systems  used  in  this  study.  What  we  would  hope  to  see  in  these 
corrected  results  would  be  Total  and  Attribute  scores  by  speaker  and 
across  speakers  that  are  statistically  equivalent  to  System  2656  and 
each  other.  An  extensive  ANOVA  across  the  individual  corrected  System 
tests  with  comparisons  to  the  individual  base  line  System  tests  shows 
no  significant  differences  at  a  95%  Confidence  Interval  (C.I.)  for 
system  means,  attribute  means  by  system,  or  individual  speaker 
attribute  scores  by  system.  As  with  the  uncorrected  System  results,  no 
significant  effect  was  indicated  between  the  methods  of  word 
replacement . 

Table  6.10  provides  a  comparison  of  all  of  the  interrupted  systems 
given  in  this  memorandum  as  corrected  for  missing  words  by  the  routine 
DRT.  These  results  are  over  the  same  system  averages  used  in  Tables 
6.2  -  6.9  and  include  the  averaged  results  for  System  2656  as  a  base 
line.  These  results  verify  the  software  packages  DRT  and  MENU.  They 
also  show  that  it  is  possible  to  achieve  meaningful  DRT  results  for  an 
interrupted  system  down  to  at  least  the  Speaker/Attribute  score. 
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Table  6.10 

COMPARISON  OF  DRT  SYSTEMS  CORRECTED  FOR  MISSING  WORDS 
(P+A  Mean  Scores  w/  S.E.) 


FEATURES 


V 

N 

SU 

SI 

G 

C 

TOTAL 

SPEAKER  NUMBER 

1 

2656 

86.2 

99.5 

93.1 

99.1 

94.8 

99.2 

95.3 

0.9 

0.2 

1 . 5 

C.5 

0.9 

0.6 

0.3 

2669 

86.5 

99.0 

93.9 

99.0 

95.0 

99.4 

95.5 

1.3 

0.3 

1.1 

0.5 

0.9 

0.3 

0.4 

2670 

86.7 

99.1 

94.8 

99.4 

94.8 

99.3 

95.7 

1.7 

0.5 

1.1 

0.4 

1.2 

0.3 

0.4 

2671 

88.2 

98.7 

95.8 

99.2 

95.7 

99.2 

96.1 

2.0 

0.7 

1.1 

0.3 

1.0 

0.3 

0.4 

2672 

85.8 

98.7 

92.7 

99.2 

97.7 

99.7 

95.6 

1.9 

0.6 

2.0 

0.4 

0.3 

0.3 

0.4 

2673 

88.3 

98.4 

94.8 

98.6 

94.6 

99.4 

95.7 

1.0 

0.9 

1.2 

0.8 

1.1 

0.4 

0.3 

2674 

88.8 

99.0 

96.4 

99.5 

95.5 

99.7 

96.5 

1.3 

0.8 

1.5 

0.4 

0.6 

0.3 

0.5 

SPEAKER  NUMBER 

2 

2656 

98.8 

99.5 

98.8 

98.8 

91.2 

98.9 

97.7 

0.4 

0.3 

0.5 

0.7 

1.2 

0.6 

0.2 

2669 

99.0 

99.0 

98.3 

98.2 

92,2 

98.8 

97.6 

0.4 

0.3 

0.6 

0.6 

1.4 

0.5 

0.2 

2670 

98.6 

98.8 

98.3 

97.8 

91.8 

99.2 

97.4 

0.4 

0.6 

0.6 

0.6 

1.1 

0.3 

0.2 

2671 

99.1 

99.6 

98.0 

97.9 

91.5 

98.6 

97.5 

0.3 

0.2 

0.6 

0.6 

1.1 

0.7 

0.2 

2672 

98.8 

97.9 

98.4 

97.9 

93.8 

99.7 

97.8 

0.4 

1.1 

0.4 

0.7 

1.6 

0.3 

0.4 

2673 

98.0 

98.7 

96.6 

97.7 

91.5 

99.5 

97.0 

1.3 

0.6 

1.2 

0.7 

1.8 

0.3 

0.5 

2674 

98.4 

99.4 

98.4 

98.7 

93.0 

98.1 

97.7 

0.7 

0.4 

0.5 

1.0 

1.4 

0.8 

0.2 

SYSTEM  2656 

PDP- 

11/44  D-2- 

A  DISK  MASTER  Q/DYN/3M 

VER  1  -  5 

(20 -NOV 

-85;22-NOV 

-85;  29 

-N0V-85;09' 

-JUL-86;11-JUL- 

86) 

SYSTEM  2669/1  ZEROED  WORDS  D2A  Q/DYN  SET-A 
VER  1  -  6  (25-JUN-86  A;25-JUN-86  B;26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

SYSTEM  2670/1  EC- 130  NOISEWORDS  D2A  Q/DYN  SET-A 
VER  1  -  6  (25-JUN-86  A;25-JUN-86  B:26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

SYSTEM  2671/1  BURST  WORDS  D2A  Q/DYN  SET-A 
VER  1  -  6  (25-JUN-86  A;25-JUN-86  B;26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

SYSTEM  2672/1  ZEROED  WORDS  Q/DYN  SET-B 
VER  1  -  3  (09-JUL-86;ll-JUL-86  A;ll-JUL-86  B) 

SYSTEM  2673/1  EC- 130  NOISE  WORDS  D2A  Q/DYN  SET-B 
VER  1  -  3  (09-JUL-86  A:09-JUL-86  B;ll-JUL-86) 

SYSTEM  2674/1  BURST  WORDS  Q/DYN  SET-B 
VER  1  -  3  (09-JUL-86  A;09-JUL-86  B; ll-JUL-86) 

NOTE!!!  S.E.  ACROSS  LISTENER  BY  SPEAKER  INTERACTIONS 
AVERAGED  ACROSS  SYSTEMS. 
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Table  6.10  (Continued) 
(P+A  Mean  Scores  w/  S.E.) 


FEATURES 


V 

N 

SU 

SI 

G 

C 

TOTAL 

SPEAKER  NUMBER  3 

2656 

98.3 

98.6 

99.8 

97.2 

93.4 

98.6 

97.7 

0.6 

0.8 

0.2 

0.8 

0.3 

0.9 

0.2 

26ij  5 

98.7 

97.3 

99.2 

96.5 

94.5 

59.0 

97.5 

0.6 

0.9 

0.2 

1.1 

1.2 

0.3 

0.2 

2670 

98.9 

97.5 

99.7 

96.7 

93.6 

99.1 

97.6 

0.4 

0.8 

0.2 

0.8 

0.9 

0.4 

0.2 

2671 

98.2 

96.7 

99.9 

97.1 

94.8 

98.7 

97.6 

0.4 

1.1 

0.1 

1.1 

1.3 

0.5 

0.3 

2672 

98.1 

98.4 

99.5 

97.2 

93.6 

100.0 

97.8 

1.2 

0.9 

0.3 

1.2 

1.4 

0.0 

0.4 

2673 

96.9 

96.6 

98.7 

96.9 

95.3 

99.0 

97.2 

1.5 

1.0 

0.4 

0.7 

1.2 

0.5 

0.3 

2674 

99.5 

97.1 

99.0 

96.4 

94.7 

99.1 

97.6 

0.3 

1.4 

0.7 

1.3 

1.4 

0.6 

0.3 

SPEAKER  AVERAGE 

2656 

94.4 

99.2 

97.2 

98.3 

93.2 

98.9 

96.9 

1.3 

0.3 

0.8 

0.4 

0.6 

0.4 

0.3 

2669 

94.7 

98.4 

97.1 

97.9 

93.9 

99.1 

96.9 

1.3 

0.4 

0.6 

0.5 

0.7 

0.2 

0.3 

2670 

94.7 

98.5 

97.6 

98.0 

93.4 

99.2 

96.9 

1.3 

0.4 

0.6 

0.4 

0.6 

0.2 

0.2 

2671 

95.1 

98.4 

97.9 

98.1 

94.0 

98.8 

97.0 

1.2 

0.5 

0.5 

0.4 

0.7 

0.3 

0.2 

2672 

94.2 

98.3 

96.9 

98.1 

95.0 

99.8 

97.1 

1.4 

0.5 

0.9 

0.5 

0.8 

0.1 

0.3 

2673 

94.4 

97.9 

96.7 

97.7 

93.8 

99.3 

96.6 

1.1 

0.5 

0.6 

0.4 

0.8 

0.2 

0.3 

2674 

95.6 

98.5 

97.9 

98.2 

94.4 

99.0 

97.3 

1.1 

0.6 

0.6 

0.6 

0.7 

0.4 

0.2 

SYSTEM  2656  PDP-11/44  D-2-A  DISK  MASTER  Q/DYN/3M 

VER  1  -  5  (20-NOV-85:22-NOV-85;29-NOV-85:09-JUL-86;ll-JUL-86) 
SYSTEM  2669/1  ZEROED  WORDS  D2A  Q/DYN  SET-A 
VER  1  -  6  (25-JUN-86  A:25-JUN-86  B;26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

SYSTEM  2670/1  EC- 130  NOISEWORDS  D2A  Q/DYN  SET-A 
VER  1  -  6  (25-JUN-86  A:25-JUN-86  B:26-JUN-86  A;26-JUN-86  B; 

02-JUL-86  A:02-JUL-86  B) 

SYSTEM  2671/1  BURST  WORDS  D2A  Q/DYN  SET-A 
VER  1  -  6  (25-JUN-86  A;25-JUN-86  B:26-JUN-86  A:26-JUN-86  B; 

02-JUL-86  A;02-JUL-86  B) 

SYSTEM  2672/1  ZEROED  WORDS  Q/DYN  SET-B 
VER  1  -  3  (09-JUL-86;ll-JUL-86  A;ll-JUL-86  B) 

SYSTEM  2673/1  EC- 130  NOISE  WORDS  D2A  Q/DYN  SET-B 
VER  1  -  3  (09-JUL-86  A;09-JUL-86  B;ll-JUL-86) 

SYSTEM  2674/1  BURST  WORDS  Q/DYN  SET-B 
VER  1  -  3  (09-JUL-86  A;09-JUL-86  B;ll-JUL-86) 

NOTE!!!  S.E.  ACROSS  LISTENER  BY  SPEAKER  INTERACTIONS 
AVERAGED  ACROSS  SYSTEMS. 
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6.3  SUMMARY  OF  THE  INTERRUPTED  DRT 


This  study  has  found  that  for  high  scoring  DRT  Systems  with 
interruptions  of  up  to  6.5  seconds  there  is  no  listener  follow  through 
or  projected  error  effect  for  the  DRT  as  conducted  at  RADC/EEV.  Also, 
trained  listeners  can  keep  s^nichronized  with  the  word  presentation 
under  the  test  conditions  and  a  modified  scoring  approach  can  provide 
meaningful  test  results  that  maintain  speaker  and  diagnostic  elements. 

Further  testing  in  this  area  could  include  measurement  of  the  Follow 
Through  Error  for  Bandwidth  Compressed  Speech  using  the  modified  word 
lists  as  inputs  and  the  testing  for  this  effect  with  partially  removed 
DRT  words.  The  testing  of  partially  removed  words  would  help  in 
defining  the  rules  for  marking  a  word  present  or  absent  from  a  test 
tape . 

The  modified  DRT  method  has  been  used  for  the  evaluation  of  the 
intelligibility  of  digitally  compressed  voice  communications  over 
severely  degraded  communications  channels.  This  effort  included 
channel  error  and  transmission  delay  simulation  along  with 
intelligibility  testing.  The  error  and  delay  simulation  software  is 
discussed  in  Reference  10. 


6.4  VOCODER  CHANNEL  DOWN  TIME 

Experiments  with  test  vocoders  and  the  simulation  of  severe  channel 
conditions  indicate  significant  periods  of  time  when  the  receiver 
experiences  unrecoverable  bit  error  states.  A  muting  algorithm  within 
the  test  vocoder  may  revert  to  silence  until  synchronization  and  a 
clear  or  acceptable  error  rate  is  recognized.  During  these  periods  of 
channel  down  time  (CDT)  the  system  has  zero  percent  intelligibility. 

The  following  describes  the  procedure  used  to  measure  the  estimated 
amount  of  time  that  zero  intelligibility  occurred  over  the  test 
vocoder  system  with  an  unknown  number  of  block  errors  injected  in  the 
digital  transmission  stream  (e.g.  Channel  Down  Time).  It  involves  the 
generation  of  analog  input  tapes  under  error  conditions,  the  measuring 
of  those  tapes  with  an  amplitude  squared  histogram  and  the  comparison 
of  those  histograms  to  one  of  the  systems  without  block  errors.  This 
procedure  was  carried  out  at  the  RADC/EEV  Speech  Processing  Facility. 


6.4.1  Data  Collection 

The  test  vocoder  system  is  set  up  in  the  same  manner  as  for  formal  DRT 
testing.  The  output  of  an  Otari  2 -track  analog  tape  recorder  is 
connected  to  the  input  of  the  transmitter.  A  bit  error  rate  box  is 
placed  in  the  digital  bit  stream  between  the  transmitter  and  the 
receiver.  The  analog  output  of  the  receiver  is  recorded  by  another 
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Otari  2- track.  Errors  are  implemented  by  a  secondary  processor  (Refs. 
7  and  10) ,  that  takes  data  off  magtape  and  sends  a  bit  stream  to  the 
bit  error  rate  box.  This  system  is  consistent  with  the  PDP-11 
implementation  used  for  the  formal  DRT  testing. 

The  output  of  a  Gaussian  noise  generator  is  passbanded  between  100  and 
5000  Hz  and  sent  to  the  Otari  input  deck.  The  gain  is  set  for  a  meter 
reading  of  -20dbVu  and  the  monitor  switch  set  to  source  to  allow 
signal  pass-through.  This  is  the  analog  input  to  the  test  vocoder 
system.  The  analog  output  is  recorded  at  -20dbVu  for  error  free  and 
block  error  conditions.  Twenty  or  more  minutes  of  output  is  recorded 
for  each  error  condition. 

These  tapes  are  then  played  back,  passbanded  again  from  100  to  5000  Hz 
and  sent  to  another  Otari  deck  where  the  filtered,  processed  white 
noise  is  set  to  -20dbVu  when  no  errors  are  present.  The  adjusted 
signal  is  passed  through  to  a  CSP-30  processor  at  EEV  for  measurement 
of  an  energy  histogram.  A  2047  bin  amplitude  squared  histogram  program 
is  used  to  take  a  twenty  minute  sample  of  each  error  condition.  The 
sampling  rate  is  10000/sec.  The  window  length  used  for  the  rms  power 
calculation  is  set  to  255.  Both  a  graph  and  a  listing  of  bin  values 
are  produced  for  each  error  condition  including  processed  white  noise 
in  the  clear.  A  sample  histogram  is  presented  in  Figure  6.2. 


ERROR1.ESS 

CBAMNEL  CBAKKEL  EMOR  EREE 

MOTES  TIME  CONDITION 


Figure  6.2  Energy  Histogram 
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6.4.2  Data  Interpretation 

The  histogram  of  the  processed,  filtered  white  noise  in  the  clear 
becomes  the  baseline  for  comparison.  This  histogram  will  show  that  all 
of  the  samples  fall  in  a  narrow  segment  of  20  bins  (54-74)  in  the 
lower  end  of  the  power  histogram.  Histograms  of  error  condition  tapes 
will  show  the  same  tendency  but  with  many  samples  in  bins  outside  of 
the  given  range.  For  all  conditions  the  vast  number  of  samples  will  be 
error  free.  Because  of  this  the  bin  with  the  greatest  number  of 
samples  in  each  histogram  can  be  used  to  correlate  the  error  condition 
with  the  error- free  condition. 

For  the  test  vocoder  in  the  above  example,  the  error  free  histogram's 
largest  point  was  bin  58.  There  were  4  lower  power  bins  and  15  higher. 
On  the  histograms  of  error  conditions,  the  sum  of  the  bins  that 
surrounded  the  largest  point  in  the  same  manner  (4  low,  15  high)  was 
calculated.  Since  all  analyses  were  calculated  over  the  same  input 
time  (20  min.),  the  total  number  of  samples  is  constant  and  gives  a 
normalization  value.  The  difference  between  the  total  samples  in  the 
error  free  state  and  the  samples  within  the  20  bin  range  of  an  error 
state  was  interpreted  to  be  a  measure  of  channel  down  time.  Given  the 
nature  of  the  error  conditions,  highly  concentrated  bursts,  samples 
above  the  20  bin  range,  were  interpreted  as  error-present  conditions 
and  lower  power  samples  as  silence  caused  by  the  muting  algorithm.  The 
histogram  difference  over  these  bins  divided  by  the  total  number  of 
samples  gave  the  percent  Channel  Down  Time. 
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CHAPTER  7 


FIELD  INTELLIGIBILITY  SYSTEM  TESTING 


7.1  IN-FIELD  DIAGNOSTIC  RHYME  TEST  (FDRT) 

There  has  long  been  a  reluctance  by  operational  personnel  to  accept 
the  results  of  laboratory  measures  of  parameters  such  as  the 
intelligibility  of  a  communication  system.  The  inability  to  totally 
model  the  environment  of  an  aircraft  in  flight  has  been  considered  a 
reason  to  doubt  results  of  tests  like  the  DRT.  The  fact  that  the 
controlled  nature  of  a  normal  DRT  is  what  limits  the  context  of  the 
test  and  allows  the  accurate  measurement  of  a  subset  of  variables  is 
not  well  understood  by  many. 

In  early  1984  the  ARCON  research  staff  felt  that  a  field  measure  of 
intelligibility  could  be  made,  but  that  the  accuracy  of  the  field 
measure  would  be  less  than  that  of  the  laboratory  measure.  A  March 
1984  AP.CON  memo  to  RADC/EEV  detailed  some  ideas  for  an  In-Place  DRT 
Procedure  using  the  TRS80  MlOO  Data  Entry  Units  (DEU) .  The  goal  was  to 
have  Speakers  and  Listeners  in  various  environments,  both  on  the 
ground  and  in  the  air,  communicating  over  actual  Air  Force  channels 
conduct  a  modified  version  of  the  DRT.  To  facilitate  this  testing,  the 
RADC/EEV  Data  Entry  System  would  be  modified  for  field  use.  Portable 
audio  tape  recorders  would  be  used  so  that  results  could  later  be 
verified  with  normal  in-house  DRTs. 

The  limitations  of  the  In-Field  DRT  include  a  limited  set  of 
non-standard  listeners  and  speakers,  no  control  of  the  transmission 
medium  and  dependence  on  other  organizations  to  provide  the  aircraft 
platforms  and  communication  systems  for  test.  The  idea  was  well 
received  and  it  was  decided  to  utilize  this  method  during  an  AF 
Electronic  Systems  Division  (ESD)  communications  system  evaluation 
scheduled  for  that  summer. 
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Over  a  four  month  period  ARGON  provided  support  to  this  evaluation 
project  that  included  the  following: 


1.  SPEAKER  Word  List  Presentation  Software 

2.  LISTENER  DRT  Software 

3.  Development  of  data  transfer  and  analysis  software 

4.  Assistance  with  experimental  design 

5.  Definition  of  the  DRT  Field  Equipment  Set 

6.  Training  of  ESD  and  RADC  speaker/listeners 

7.  Generation  of  field  training  procedures  of  operational 
speaker/listeners 

8.  Development  of  Procedures  for  processing  results 

9.  Processing  of  Field  data 

10.  Verification  analysis  of  the  In-Field  DRT 


Extensive  field  tests  and  surveys  were  conducted  by  ESD  and  RADC 
personnel.  An  accelerated  testing  schedule  and  untested  software 
caused  early  problems.  Although  Field  test  procedures  were  well 
thought  out  and  documented,  field  data  was  incomplete  because  of 
equipment  malfunctions  ranging  from  voice  processor  failures  to 
aircraft  assignment  difficulties.  Program  schedules  were  unrealistic 
and  baseline  data  was  unattainable  from  some  listener/speakers  because 
of  their  other  commitments.  All  of  these  problems  resulted  in  the 
inability  to  verify  or  even  relate  the  field  test  results  to  in-housc 
tests  made  on  audio  recordings  returned  from  the  field. 

The  basic  experimental  design,  DRT  Field  Equipment  Set,  test  software 
and  field  procedures  as  specified  for  this  effort  were  not  at  fault  in 
the  inability  to  verify  the  relationship  between  In-Field  and  in-house 
DRTs .  The  mixed  task  objectives  of  validating  the  test  method  and 
evaluating  a  specific  communication  system  were  primarily  responsible 
for  the  failure  to  accomplish  either.  They  led  to  unclear  areas  of 
responsibility  and  a  low  priority  for  the  DRT  component  of  the  overall 
field  test. 

To  facilitate  this  testing,  the  system  used  for  taped  DRT's  at  the 
RADC/EEV  Speech  Lab  was  modified.  This  report  will  document  the 
adaptations  made  to  existing  software  in  support  of  this  effort.  The 
source  code  for  the  Field  DRT  software  package  is  available  from 
RADC/EEV.  The  DRT  field  equipment  set  used  in  the  ESD  survey  will  be 
discussed.  The  transfer  of  data  and  programs  to/from  the  Data  Entry 
Units  was  cumbersome.  Later,  a  new  method  for  downloading  programs  to 
the  DEUs  was  developed.  This  method  will  also  be  discussed. 
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7.1.1  Software  Development 


One  of  the  major  differences  between  taped  and  live  DRT's  is  the 
possibility  of  loss  of  communication  between  Speaker  and  Listener(s) 
during  live  tests.  This  eventuality  must  be  allowed  for  by  the 
software,  and  recovery  must  be  relatively  straightforward.  Also,  it 
must  be  recognized  that  in  a  communications  setup  involving 
"push- to- talk"  microphones,  for  relatively  long  periods  of  time  the 
Speaker  will  be  unable  to  verify  with  the  Listener(s)  that  he  is  still 
in  fact  being  heard.  Finally,  some  method  must  be  provided  for  th® 
Speaker  to  be  presented  with  the  proper  words  to  speak,  with  the  right 
time  interval  between  words. 

These  considerations  were  dealt  with  by  providing  two  programs  to  be 
run  on  Radio  Shack  TRS-80  Model  100  portable  computers.  These  are  the 
same  machines  used  with  the  taped  DRT  sessions.  The  two  programs  are 
LISTEN. BA  and  SPEAK. BA.  LISTEN  is  a  modified  version  of  the  DRT. BA 
program  used  for  taped  DRT's,  while  SPEAK  was  written  for  this 
project.  Both  programs  include  a  facility  for  repeating  a  page  of  a 
DRT  wordlist  (i.e.,  58  word-pairs)  in  the  event  of  loss  of 
communications . 


LISTEN  Program  -  LISTEN  is  a  modified  version  of  the  program  DRT, 
which  was  originally  written  for  the  taped  DRT  sessions.  Since  DRT  is 
fully  documented  elsewhere,  this  section  will  describe  only  the 
changes  made. 

The  major  difference  between  the  two  programs  is  the  inclusion  in 
LISTEN  of  the  facility  to  restart  a  page  of  a  wordlist.  The  assumption 
made  is  that  the  Speaker  will  hold  down  his  "push- to- talk"  button  for 
the  duration  of  one  page  (58  words)  of  the  current  list.  He  will  then 
ask  the  Listener(s)  whether  or  not  all  of  the  page  was  heard.  If 
communication  was  interrupted,  the  Listener(s)  can  request  a  repeat  of 
that  page .  Otherwise  the  test  continues  with  the  next  page . 

If,  during  the  course  of  a  page  of  a  DRT,  communication  is 
interrupted,  the  Listener  need  not  finish  that  page  in  order  to  have 
the  program  restart  it.  At  any  time,  the  Listener  can  hit  the  Left 
Arrow  key  on  the  MlOO  keyboard.  This  is  the  signal  to  LISTEN  that  the 
current  page  should  be  restarted.  The  program  immediately  jumps  to  a 
subroutine  which  displays  the  approximate  number  of  seconds  left 
before  the  Speaker  finishes  the  page.  At  any  time,  the  Listener  can 
restart  the  page  by  hitting  any  key.  Presumably  this  will  be  done  when 
communication  is  re-established  and  the  Speaker  is  made  aware  of  the 
necessity  to  repeat.  This  procedure  can  be  repeated  any  number  of 
times  for  any  one  page  of  the  test.  Once  a  page  has  been  accepted  as 
complete,  that  data  is  stored,  and  the  page  is  no  longer  repeatable. 
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There  are  only  three  other  differences  between  LISTEN. BA  and  DRT.BA. 
The  first  is  that  in  LISTEN. BA  no  assumption  is  made  regarding  the 
number  of  systems  or  speakers  to  be  heard.  Therefore,  before  each 
booklet  is  started,  the  Listener  is  prompted  for  one  wordlist  number 
and  one  speaker  I.D.  In  DRT.BA,  it  is  assumed  that  three 
speaker/wordlist  combinations  will  be  run  at  a  time,  so  all  three  are 
prompted  for  at  once. 

Another  minor  change  was  that,  while  the  same  Listener  can  be  expected 
to  use  the  same  MlOO  for  an  entire  day's  testing  with  taped  DRT's,  the 
same  is  not  necessarily  the  case  for  live  tests.  Therefore,  instead  of 
storing  responses  in  a  file  tagged  with  the  Listener's  ID  number  as  in 
DRT.BA,  LISTEN. BA  appends  each  booklet's  answers  to  one  file, 
ANSWER . DO . 

The  last  difference  between  the  two  programs  is  that  the  two  original 
wordlist  files  used  by  DRT.BA,  WORD. DO  and  PROBE. DO,  are  broken  up 
into  first  and  second  halves  for  LISTEN. BA  (WORDl.DO,  W0RD2.D0, 
PROBl.DO,  PR0B2.D0).  This  was  done  to  speed  up  the  process  of  backing 
up  to  the  top  of  a  page  when  a  restart  is  requested. 

Except  for  the  differences  noted,  LISTEN. BA  operates  in  the  same  way 
as  DRT.BA.  Word  pairs  are  displayed  in  the  same  manner,  responses  are 
accepted  and  stored  in  the  same  format,  and  the  same  options  regarding 
training  states,  reverse  video,  etc.,  are  present. 


SPEAK  Program  -  The  program  SPEAK. BA  was  written  specifically  for  this 
application.  It  is  based  on  DRT.BA,  but  its  purpose  is  to  display  a 
list  of  DRT  word  pairs  for  a  DRT  speaker.  SPEAK  allows  the  user  to 
select  the  wordlist  key  from  a  set  of  such  keys  stored  in  file 
KEYLST.DO.  The  time  interval  between  words  may  also  be  chosen,  with  a 
default  of  1.3  seconds.  SPEAK  then  displays  the  indicated  word  pairs, 
scrolling  through  the  list  at  the  specified  rate.  At  the  end  of  a 
page,  the  Speaker  is  given  the  opportunity  to  either  continue  to  the 
next  page  or  repeat  the  page  just  completed.  At  this  point  the  Speaker 
inquires  of  the  Listener(s)  whether  or  not  he  should  continue,  and 
proceeds  with  the  appropriate  page.  This  process  continues  until  the 
end  of  the  current  list,  and  can  be  repeated  indefinitely. 

SPEAK. BA  consists  of  four  sections:  Initialization,  Prompt/Response, 
Wordlist  Display,  and  Subroutines.  These  are  outlined  below. 


Initialization  Section  -  This  section  is  executed  at  program  startup. 
The  screen  is  cleared,  the  maximum  nximber  of  files  to  be  used 
declared,  string  space  is  allocated  and  initialized,  memory  is 
allocated  for  the  MCODE.CO  machine  language  subroutines  and  MCODE  is 
loaded,  and  several  variables  are  initialized. 
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Prompt/Response  Section  -  In  the  Prompt/Response  Section,  the 
parameters  of  the  test  are  input  by  the  user  and  stored  by  the 
program.  All  answers  given  in  this  section  are  echoed  back  to  the 
Speaker  for  verification.  If  the  Speaker  indicates  the  data  is 
correct,  the  next  question  is  asked.  Otherwise  the  last  question  is 
asked  again. 

The  Speaker  is  asked  first  whether  the  list  to  be  used  is  the  Probe  or 
Data  list.  The  response  must  be  either  "P"  or  "D" ,  or  "E"  for  exit. 
Then  the  Speaker's  ID  is  entered.  The  expected  format  is  two  initials. 
Next  a  list  of  available  keys  is  displayed.  The  keys  determine  which 
word  of  a  given  word  pair  is  to  be  spoken.  The  Speaker  is  asked  to 
select  one  of  the  keys,  and  this  is  checked  for  validity  against  the 
list  displayed.  If  valid,  execution  proceeds  to  the  next  question, 
otherwise  the  Speaker  is  prompted  for  a  new  key. 

Once  the  key  has  been  selected,  the  Speaker  is  given  the  option  of 
changing  the  delay  period  between  words.  The  default  of  1.3  seconds  is 
selected  by  entering  a  carriage  return,  but  any  period  down  to  .52 
seconds  may  be  selected.  This  choice  is  made  available  for  training 
purposes . 

After  all  of  the  above  has  been  entered,  the  Speaker  is  instructed  to 
read  a  standard  DRT  prologue  message  containing  the  Speaker's  ID,  the 
wordlist  key,  and  the  date.  When  the  speaker  hits  carriage  return,  the 
program  branches  to  the  wordlist  display  section. 


Wordlist  Display  Section  -  This  is  the  section  which  displays  the 
wordlist  and  changes  the  "current"  word  pair  at  the  specified 
interval.  The  LCD  screen  consists  of  8  40-character  lines.  At  the 
beginning  of  a  page,  the  wordlist  and  page  number  are  displayed  oi. 
line  1.  From  that  point  on  however,  lines  1  -  4  are  blank.  Line  5 
contains  the  "current"  word  pair.  This  pair  has  the  word  which  is  to 
be  spoken  on  the  left  side,  highlighted  in  reverse  video.  Line  6  is 
blank,  and  lines  7  and  8  contain  two  upcoming,  or  "future"  word  pairs, 
all  in  normal  video. 

At  the  specified  time  interval,  the  display  scrolls  upward.  The 
contents  of  line  5  disappear,  and  the  contents  of  line  7  are  moved  to 
line  5,  with  the  lefthand  word  re-written  in  reverse  video.  This 
becomes  the  next  "current"  word  pair.  At  the  same  time,  line  8's 
contents  move  to  line  7,  and  a  new  pair  appears  on  line  8.  Lines  1-4 
and  line  6  are  always  blank. 

This  scrolling  process  continues  until  the  end  of  a  page  is  reached  (a 
page  is  defined  as  58  word  pairs).  At  this  point,  a  message  is 
displayed  indicating  end-of-page,  and  the  Speaker  is  instructed  to 
obtain  Listener  acknowledgement  before  continuing.  If  the  page  just 
completed  is  to  be  repeated,  the  Speaker  enters  an  "R" ;  to  proceed 
with  the  next  page,  the  Speaker  enters  a  "P" .  This  process  continues 
until  the  4 -page  "book"  is  completed.  The  program  then  jumps  back  to 
the  Prompt/Response  Section  and  starts  again. 
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Subroutine  Section  -  This  section  contains  various  subroutines  called 
in  the  other  sections.  Subroutines  exist  to  display  the  End-of-Page 
message  and  input  the  option  to  proceed  or  repeat,  to  form  the  next 
word  pair  from  the  keylist  and  wordlist,  to  display  the  KEYLST.DO  file 
and  obtain  the  Speaker's  selection,  and  to  check  that  selection  for 
validity . 


KEYLST ■ DO  File  Format  -  The  file  KEYLST.DO  is  needed  for  correct 
operation  of  SPEAK. BA.  This  file  contains  the  list  of  available  keys 
for  use  in  deciding  which  word  of  the  "current"  pair  should  be  spoken. 
This  file  consists  of  63-byte  records  which  have  a  5-byte  key 
identification  followed  by  a  58-byte  key.  The  key  ID  is  of  the  form 
"Wnnna" ,  where  "nnn"  is  a  3  digit  number,  and  the  "a"  is  either  "A"  or 
"B" .  The  key  is  just  a  string  of  zeroes  and  ones  indicating  whether 
the  left  hand  or  righthand  word  should  be  spoken.  This  key  is  used 
once  per  page.  On  pages  one  and  four,  a  one  indicates  that  the  first 
word  of  a  pair  should  be  spoken,  while  a  zero  means  the  reverse.  On 
pages  2  and  3,  a  one  indicates  that  the  second  word  of  a  pair  should 
be  spoken,  while  a  zero  shows  that  the  first  word  of  a  pair  should  be 
spoken. 


Modifications  To  The  "MCODE"  Machine  Language  Subroutines  -  SPEAK. BA 
and  LISTEN. BA  use  some  of  the  machine  language  subroutines  written  for 
the  DRT.BA  program.  In  the  case  of  the  LISTEN  program,  one 
modification  was  necessary  to  one  of  MCODE' s  routines.  The  original 
DRT.BA  program  passed  three  integers  to  the  keyboard  polling  routine. 
These  were  used  as  flags  to  determine  whether  the  Listener  made  a 
left/right  word  choice,  or  whether  a  change  in  a  past  selection  was 
indicated,  or  whether  the  position  of  the  change  indicator  should  be 
moved.  The  status  of  these  three  flags  indicated  what  action  the 
program  should  take.  LISTEN. BA  uses  a  fourth  flag  which  indicates 
whether  or  not  the  current  page  should  be  restarted.  The  machine 
language  subroutine  INIT  was  modified  to  store  the  location  of  this 
fourth  flag,  and  the  KBD  keyboard  monitoring  routine  was  changed  to 
recognize  this  key  and  report  its  status. 
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7.1.2  Equipment 

ARGON  defined  the  following  items  for  the  DRT  Field  Equipment  Set 
during  the  1984  AF  ESD  communications  system  evaluation: 

1.  4  Sony  TC-D5M  Audio  Cassette  recorders 

2.  2  Sony  ECM-16T  Electret  Condenser  Microphones 

3.  GenRad  1565  Sound  Level  Meter 

4.  4  Radio  Shack  TRS-80  Model  100  computers 

5.  2  Radio  Shack  Minisette-9  Data  recorders 

6.  Speaker/Listener  Training  Program  on  Audio  cassette 

7 .  Backup  Programs  for  MlOOs  on  Data  cassette 

8 .  Headphones 

9.  Batteries  (AA's  and  C's) 

10.  2  Interface  boxes 

11 .  Blank  Audio  and  Data  cassette  tapes 

12 .  Cables ,  and  Adaptors 


One  audio  cassette  recorder  was  interfaced  to  the  communication  system 
at  each  station  to  record  either  transmit  or  receive  lines  at  both  the 
pre  and  post  intercom  position  (Fig.  7.1).  Adaptor  cables  and  a  simple 
interface  box  designed  for  this  specific  application,  allowed  test 
participants  to  easily  switch  and  adjust  channels  and  recorder  input 
levels.  During  transmission,  the  speaker's  recorder  would  be  accessing 
the  headset  output  and  the  intercom  output.  When  receiving,  a 
listener's  recorder  would  access  the  vocoder  output  and  the  intercom 
output.  A  backup  recorder  was  available  at  both  stations.  Tapes 
recorded  on  site  could  then  be  evaluated  at  the  RADC/EEV  Speech  Lab  by 
the  in-house  listening  crew. 


Figure  7.1  Field  DRT  Recording  Configuration 
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Recordings  of  the  ambient  noise  environment  were  generated  using  a 
separate  (backup)  recorder  and  the  two  Sony  mics.  Sound  level 
measurements  were  also  made  with  the  GenRad  meter  at  and  "C" 
weightings  to  coincide  with  these  noise  recordings. 

Two  Model  100  computers  were  brought  to  each  station.  One  machine 
configured  for  speaking,  the  other  for  listening.  The  Data  recorders 
were  used  to  store  listener  response  data.  The  Model  100  computers' 
listener  data  files  were  downloaded  onto  cassette  at  the  end  of  each 
test  day.  Backup  copies  of  all  the  necessary  MlOO  programs  were 
recorded  at  RADC/EEV  prior  to  field  testing  on  Data  cassette  and  these 
could  be  uploaded  to  the  MlOOs  from  the  Data  recorders. 

An  audio  cassette  copy  of  the  training  series  used  for  the  RADC/EEV 
Speech  Lab's  in-house  listening  crew  was  brought  to  train  field 
listener/speakers.  A  large  supply  of  high-quality  batteries  was  needed 
to  insure  ample  operating  time  for  all  portable  equipment.  Headphones 
allowed  the  monitoring  of  tape  decks  during  preliminary  testing  and 
throughout  the  DRT  testing  and  noise  measurement  efforts  to  maintain 
quality  tape  reco;"’ings. 


7.1.3  Program  Download  Procedure 

Software  and  data  can  be  transferred  to/from  a  Model  100  DEU  through 
either  the  RS232  port,  the  cassette  recorder  connector  or  a  floppy 
disk  drive.  The  field  tests  depended  entirely  on  cassette  recorder 
data  and  program  transfer.  This  proved  to  be  time  consuming  and 
unreliable.  The  incorporation  of  a  Tandy  Disk/Video  Interface  should 
solve  this  problem,  however,  an  even  faster  method  to  initiate  MlOOs 
has  been  developed  and  will  be  documented  here. 

Once  a  single  Model  100  is  loaded  with  the  proper  Speaker  programs  and 
a  second  is  loaded  for  the  Listener,  a  program  to  cross -load  from 
these  MlOOs  to  others  is  faster  than  loading  each  MlOO  from  the  RS232 
or  cassette  recorder  ports.  A  pair  of  programs  called  MLOAD  and  XLOAD 
allow  cross -loading  from  one  MlOO  to  another  via  the  RS-232C 
communications  ports.  The  two  machines  must  be  connected  by  a 
"null -modem"  cable.  When  this  program  completes  execution,  the 
"destination"  machine  has  an  exact  copy  of  the  "source"  machine's  RAM, 
including  all  BASIC,  document,  and  machine -language  files.  The  program 
does  not  set  the  clock  on  the  destination  machine.  Each  machine  keeps 
a  count  of  any  comm  errors.  The  BASIC  program  will  print  the  final 
error  counts  on  both  machines  to  verify  that  a  load  was  successful,  or 
advise  that  the  load  failed  and  should  be  reaccomplished. 
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To  load  one  Model  100  from  another  first  ensure  that  the  "source" 
machine  has  a  copy  of  the  BASIC  program  "XLOAD.BA"  in  a  RAM  file.  Then 
connect  both  machines  with  an  RS-232C  "null -modem"  cable  in  the 
RS-232C  connectors  on  the  back  of  both  computers.  Place  both  machines 
in  the  BASIC  command  mode  by  selecting  BASIC  from  the  main  menu.  On 
the  source  machine  type : 


RUN  "COM;98N2E"<enter> 

On  the  destination  machine  type: 

LOAD  "XLOAD"<enter> 

SAVE  "COM;98N2E"<enter> 
RUN  l<enter> 


Wait  45  seconds  and  then  check  the  error  counts  printed  by  both 
machines.  If  both  error  counts  are  zero,  then  the  cross -load  was 
successful.  Otherwise,  repeat  the  above  procedure  until  both  machines 
report  zero  errors.  The  program  should  not  fail,  so  repeated  failures 
may  indicate  a  hardware  problem  in  one  or  both  machines. 

The  cross -load  program  does  not  require  that  the  destination  machine 
be  zeroized,  only  that  it  not  be  "hung-up"  or  "crashed".  The  program 
does  not  initialize  the  Date/Time  Clock  on  the  destination  machine. 
DATE$ ,  DAY$  and  TIME$  may  be  set  in  the  normal  manner  either  before  or 
after  cross- load  is  completed. 

The  source  code  for  the  programs  MLOAD  and  XLOAD  are  available  from 
RADC/EEV. 
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CHAPTER  8 


DRT/DAM  MASTERING  PROGRAM 


ARGON  Corporation  has  supported  the  DoD  Digital  Voice  Processing 
Consortium  (DDVPC)  in  their  Test  and  Evaluation  efforts  for  the  past 
five  years.  As  part  of  this  effort  ARGON  personnel  have  been 
responsible  for  the  development  of  a  version  of  the  DRT/DAM  input 
master  library  on  digital  audio.  The  Test  and  Evaluation  Facility  at 
RADC/EEV  has  been  used  for  the  research,  development  and 
implementation  of  a  system  to  measure  Speech- to-Noise  ratios  and 
electrical  levels  on  audio  tape  (Refs.  7  and  11).  This  method  has  been 
used  to  equalize  the  DRT/DAM  master  tape  library  in  conjunction  with 
converting  the  library  to  digital  audio  on  video  tape.  The  end  result 
is  a  library  with  equalized  presentation  levels  and  measured 
speech- to-noise  values  for  all  speaker/list/channel  combinations. 
Digital  audio  has  allowed  the  library  to  be  precisely  and  uniformly 
edited.  Each  speaker  has  been  listed  with  its  beginning  time  code 
value  to  make  location  within  the  master  a  simple  task.  Digital  Audio 
has  captured  the  DRT/DAM  library  accurately  within  its  20  to  20,000 
Hz.  flat  frequency  response  and  has  also  given  the  library  a  better 
storage  life  than  its  analog  predecessor.  This  new  media  also  allows 
the  creation  of  digital  working  copies  that  lose  no  quality  even  over 
repeated  generations.  ARGON  has  been  responsible  for  the  equalization 
and  digital  mastering  of  the  entire  DDVPC  master  tape  library  over  the 
past  four  years.  This  project  is  now  complete. 


8.1  DIGITAL  MASTER  LIBRARY 

Master  tapes  of  the  DRT/DAM  digital  audio  library  reside  on  U-matic 
videocassette.  Each  tape  contains  a  single  three  speaker  group  of  a 
common  gender  in  a  particular  noise  environment.  Speaker/lists  are 
recorded  on  two  channels  using  two  different  microphones.  Most  often, 
Channel  Left  will  be  the  resident  microphone  of  the  particular  noise 
platform  and  Channel  Right  will  be  the  Altec  659A  dynamic  microphone. 
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The  Altec  is  a  high-quality,  full  frequency,  omni-directional  mic . 
Appendix  A  is  a  complete  listing  of  the  DRT/DAM  digital  audio  input 
tape  library.  Major  groups  are  organized  by  noise  environment  and 
microphones  and  sub-grouped  by  speaker  sex.  Included  are  speakers, 
lists  and  speech-to-noise  ratios  for  each  speaker/list/channel . 
Information  regarding  the  procedures  and  equipment  used  for  the 
original  production  of  the  analog  roaster  tapes  can  be  found  in 
References  12  and  13. 


8.1.1  Tape  Fo rmat 

The  U-matic  digital  DRT/DAM  master  series  tapes  follow  this  format; 


1)  30  seconds  of  silence  (SMPTE  time  code  0-30  sec.) 

2)  Reference  tone:  IKHz  at  +5dB  Vu 

3)  Reference  tone:  5KHz  at  OdB  Vu 

4)  Speaker  1  DAM  list  announcement 

5)  Speaker  1  DAM  list 

6)  Speaker  2  DAM  list  announcement 

7)  Speaker  2  DAM  list 

8)  Speaker  3  DAM  list  announcement 

9)  Speaker  3  DAM  list 

10)  Reference  tone:  IKHz  at  +5dB  Vu 

11)  Reference  tone;  5KHz  at  OdB  Vu 

12)  Speaker  1  DRT  list  announcement 

13)  Speaker  1  DRT  list 

14)  Speaker  2  DRT  list  announcement 

15)  Speaker  2  DRT  list 

16)  Speaker  3  DRT  list  announcement 

17)  Speaker  3  DRT  list 


DAM  lists  are  mastered  once  for  each  speaker  group/environment.  Each 
master  containing  DRT  scramblings  for  a  particular  speaker 
group/environment  has  an  exact  digital  dub  of  the  DAM  lists.  This 
assures  consistent  input  material  for  DAM  test  generation  regardless 
of  which  DRT  scrambling  is  used.  Each  list  is  separated  from  the  next 
announcement  by  10  seconds.  Each  master  has  its  own  index  complete 
with  list  numbers  and  clock  values  to  enable  use  of  the  auto  locator 
function  to  find  desired  sections. 


8.1.2  Label  Format 

The  titles  of  the  digital  audio  library  masters  were  developed  by 
RADC/EEV  and  differ  from  their  analog  predecessors  only  in  the  first 
letter  prefix.  Table  8.1  is  a  comparison  of  old  and  new  titles. 
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Table  8.1 
Master  Tape  Titles 


ANALOG  VS.  DIGITAL  DRT/DAM  MASTER  TITLES 


Noise  Environment 

Analog 

Digital 

Quiet 

(males) 

(females) 

R-l-A 

E5-1.1-A 

D-l-A 

D5-1.1-A 

Helicopter 

(males) 

(females) 

G-2-A 

G2-1.1-C 

N-2-A 

N2-1.1-C 

Shipboard 

(males) 

(females) 

G-3-A 

G3-1.1-C 

N-3-A 
N3-1. 1-C 

Office 

(males) 

(females) 

G-4-A 

G4-1.1-C 

N-4-A 

N4-1.1-C 

E-3A  Aircraft  (males) 
(females) 

Kl-l.l-A 

Kl-l.l-C 

Jl-l.l-A 

Jl-l.l-C 

Destroyer 

(males) 

(females) 

K5-1.1-A 

K5-1.1-C 

J5-1.1-A 

J5-1.1-C 

Hcl I  copter 
Carrier 

(males) 

(females) 

K6-1.1-A 

K6-1.1-C 

J6-1.1-A 

J6-1.1-C 

Other  Noise  platforms  previously  using  the  "K"  prefix 
(P-3C  Aircraft,  Jeep,  Tank,  and  E-4B  Aircraft)  have 
also  been  converted  to  "J"  prefixes. 


The  title  of  a  Digital  DRT/DAM  master  is  displayed  on  the  spine  of  the 
U-matic  tape  box.  On  the  front  the  title  is  redisplayed  along  with  a 
description  of  the  noise  environment,  the  microphone  channel 
assignments,  the  number  and  gender  of  the  speaker  set,  and  the  master 
serial  number.  Front  Labels  appear  in  the  following  format: 


DRT  JlO-l.l-A  E4B  BATTLESTAFF 
THREE  SPEAKER  MALE 
CH  L.  ROANWELL  240100001 
CH  R.  ALTEC  659A  DYN. 

SERIAL  #  00000145 

SEE  INSERT  FOR  DETAILS 
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On  the  inside  front  cover  of  each  tape  box  is  the  insert.  This  gives 
the  speaker  initials,  the  lists  used  and  the  time  code  values  for  each 
of  the  lists  and  reference  tones.  These  inserts  appear  in  the 
following  format: 


DRT  TAPE  JlO-l.l-A 

E4B  AIRCRAFT,  BATTLESTAFF  COMPARTMENT 


TIME  CODE 

MIN: SEC  SPEAKER  DAM  LIST 
00:30  REFERENCE  TONES 
00:50  RH(M)  6A 

02:20  JE(M)  3A 

03:50  CH(M)  4A 

05:20  REFERENCE  TONES 

SPEAKER  DRT  LIST 
05:40  RH(M)  323A 

11:50  JE(M)  314B 

18:00  CH(M)  309B 


DIGITAL  COPIES  OF  ANALOG  MASTERS 

The  footnote  was  included  to  differentiate  between  lists  copied  from 
the  analog  library  and  those  rerecorded  direct  to  digital  (as  in  the 
case  of  some  of  the  quiet  environment  DAM  lists). 

The  DRT/DAM  digital  master  library's  primary  use  in  the  immediate 
future  will  be  for  the  production  of  high  quality  working  copies. 
Direct  digital  copies  will  retain  all  the  consistency  and  response  of 
the  original  masters.  Analog  working  copies  will  also  show  improvement 
in  speaker  level  consistency  and  universal  formatting.  The  RADC/EEV 
facility  currently  has  the  capability  of  generating  reel-to-reel 
analog  and  Beta  videocassette  digital  working  copies.  Past  experience 
shows  that  VHS  videocassette  copies  are  easily  produced  as  well. 


8.2  MASTERING  SYSTEM 

Original  master  tapes  are  played  back  on  an  AMPEX  1/2"  four  track 
recorder.  The  two  tracks  designated  to  be  digitized  are  sent  through 
independent  attenuators  that  provide  up  to  111  db  of  passive  padding 
in  .1  db  increments.  That  output  runs  to  the  PCM  mastering  system.  The 
analog  output  of  that  system  flows  to  an  Otari  MX50-50  recorder.  This 
recorder's  output  is  sent  to  a  passive  band-pass  filter  set.  A 
transformer  is  inserted  to  provide  line  isolation  and  the  signal  flows 
through  it  to  the  PDP  11/34  (see  Fig.  8.1). 
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Figure  8.1  RADC/EEV  Digital  Audio  Mastering  System 


8.2.1  Analog  Source 

The  analog  tape  recorder  used  for  the  playback  of  the  original  analog 
DRT/DAM  masters  is  an  Ampex  440-C.  This  machine  matches  the  one  used 
in  the  recording  of  these  masters.  A  headphone  monitoring  jack  is 
available  for  each  channel. 
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8.2.2  Digital  Audio 

The  PCM  mastering  system  consists  of  a  PCM- 100  digital  audio 
processor,  a  DAE- 1100  digital  audio  editor,  and  two  BVU  800B 
professional  U-matic  videocassette  recorders.  This  system  converts  the 
two  analog  channels  to  digital,  records  onto  videocassette,  edits,  and 
outputs  two  channel  analog.  A  stereo  headphone  monitoring  jack  is 
available.  All  components  of  the  digital  audio  editing  system  are 
interconnected  according  to  instructions  and  illustrations  in  the 
DAE-1100  operation  and  maintenance  manual  (Ref.  14). 

The  four  switches  on  the  face  of  the  Digital  In  Out  Unit  within  the 
PCM- 100  are  to  be  set  as  follows  for  recording  analog  information 
entered  via  the  Analog  In  cannon  plugs.  This  also  allows  output  from 
the  Analog  Out  cannon  plugs.  The  output  occurs  whether  the  record  deck 
is  engaged  or  not. 


EXT  SYNC :  INT 
REC  DATA:  INT 
PB  DATA:  INT 
REC  MASTER:  REC 

The  dubbing  switch  of  the  Analog  Digital  Convert  Unit  is  in  the  off 
position.  The  emphasis  switch  is  set  to  off  during  the  initial 
calibration  and  engaged  during  measurement  and  recording  of  speech 
material . 

Emphasis  is  a  form  of  high-end  boost  that  conditions  the  incoming 
analog  signal  to  ease  conversion  to  digital.  By  amplifying  the  upper 
audio  frequencies,  samples  of  these  smaller  waveforms  are  more 
realistically  represented.  Upon  playback,  circuitry  automatically 
detects  the  presence  of  emphasis  and  readjusts  the  analog  output 
accordingly.  The  increase  in  input  signal  strength  due  to  emphasis  is 
reflected  in  the  PCM- 100  program  meters,  although  the  output  has  been 
equalized.  This  is  to  guard  against  saturation  of  the  A/D.  In  light  of 
this,  all  reference  tones  have  been  adjusted  and  recorded  without 
emphasis  to  allow  a  direct  and  accurate  meter  reading. 

The  Digital  Audio  Mastering  System  converts  the  analog  signals  into 
EIAJ  14-bit  digital  with  SMPTE  time  codes.  Sampling  frequency  is 
44.056  kHz.  Quantization  is  selectable  between  14-bit  and  16 -bit 
linear.  Coding  formats  are  the  EIAJ  standard  for  14-bit  and  a  Sony 
16-bit  format.  In  Air  Force  applications,  the  standard  14-bit  format 
is  used.  This  allows  the  playback  of  these  recordings  through  all  PCM 
processors  using  the  EIAJ  decoding.  Although  the  16-bit  format  allows 
4db  more  dynamic  range  and  slightly  less  harmonic  distortion,  the 
error  correction  circuitry  that  the  extra  two  bits  were  taken  from  is 
more  important  for  successful  storage  and  dubbing  (Ref.  15). 
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Sony  PCM  digital  audio  systems  boast  a  near  flat  response  from  20  to 
20,000  Hz.  This  specification  wa.s  confirmed  with  spectral  analysis  at 
the  RADC/EEV  Speech  Lab.  Samples  of  input  and  output  signals  showed  no 
significant  difference  in  the  frequency  spectra  of  the  two  sources  in 
the  20-20K  Hz.  bandwidth. 

Digital  audio  information  is  converted  to  a  video  signal  and  recorded 
onto  U-matic  tape.  One  of  the  two  analog  bands  on  the  cassette  is  used 
for  SMPTE  time  code.  The  PCM  system  for  videocassette  overlaps  channel 
information  in  such  a  way  that,  once  recorded,  it  is  impossible  to 
digitally  separate  the  two  tracks.  The  relative  balance  between  left 
and  right  tracks  at  the  time  of  recording  is  preserved.  Only  by 
converting  to  analog  may  the  signals  be  exclusively  manipulated.  The 
digital  mastering  system  also  allows  the  user  to  inscribe  an  8  digit 
number  within  the  digital  code.  RADC/EEV  has  elected  to  use  this  User 
Bits  function  to  inscribe  a  serial  number  for  each  of  the  DAM/DRT 
master  modules. 

Tapes  generated  on  the  PCM  Digital  Audio  Mastering  System  are 
compatible  with  any  other  processor  conforming  to  the  EIAJ  14 -bit 
standard  or  Sony's  16-bit  format.  However,  because  the  storage  medium 
used  is  videocassette,  another  variable  arises,  that  is  the  video 
waveform.  Digital  tapes  originating  from  most  European  organizations 
will  not  be  compatible  with  RADC/EEV  equipment  due  to  Europe's  vastly 
different  television  waveform  standard  (Ref.  16). 


8.2.3  Analog  Output 

The  two  analog  channel  outputs  of  the  PCM- 100  meet  a  switching  box 
just  prior  to  the  input  of  the  Otari  analog  tape  recorder.  The  throw 
of  the  switch  determines  which  channel  is  to  be  monitored  and 
measured. 

The  Otari  MX50-50  two- track  analog  recorder  is  primarily  used  as  a 
line  amplifier.  The  deck  is  set  to  monitor  line  input  and  the  input 
gain  control  is  calibrated  in  order  for  the  PDP  11/34  to  properly 
measure  the  EPL  and  S/N  ratios  (see  Reference  Tones) .  The  Otari  also 
provides  a  useful  point  to  monitor  the  audio  signal  before  filtering. 
A  speaker  has  been  included  in  the  system  configuration  for  this 
purpose . 

The  bandpass  filter  set  used  as  anti-aliasing  filters  for  the  PDP 
11/34  A/D  is  manufactured  by  Allison  Labs  and  is  model  2-BR.  The  low 
cutoff  is  set  at  150  Hz.  and  the  high  cutoff  is  at  4000  Hz.  Before 
reaching  the  PDP  11/34' s  A/D  the  signal  passes  through  a  transformer 
for  the  purpose  of  line  isolation. 
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The  POP  11/34  has  an  A/D  with  an  8000/sec.  sampling  rate.  Sampling 
begins  when  the  space  bar  of  the  terminal  is  pressed  and  continues  for 
forty  seconds.  Sampling  begins  directly  after  the  first  word  or 
sentence  in  the  DRT  or  DAM  list.  Results  are  output  to  the  terminal 
screen  with  an  option  to  dump  to  a  printer. 

Measurements  of  the  analog  master  tape  library  show  widely  varying 
presentation  levels  not  only  across  different  tapes,  but  also  across 
different  speakers  within  a  single  master.  It  is  not  uncommon  to  find 
differences  on  the  order  of  6dB  between  one  speaker  and  the  next,  even 
using  the  same  microphone  channel.  The  digital  library  has  equalized 
presentation  levels  to  within  0.5dB. 

8.3  MASTERING  PROCEDURES 

The  first  step  in  the  digital  mastering  of  the  DRT/DAM  library  is  to 
calibrate  all  parts  of  the  system  for  unity  gain.  Initial  measurements 
of  the  analog  masters  are  made  by  allowing  the  signal  to  flow  through 
the  entire  system.  An  accurate  result  depends  on  equal  levels 
throughout  the  system.  After  preliminary  measurements , the  channel 
attenuators  are  adjusted  and  new  measurements  are  made.  Once  the 
EPL  figure  is  reached  the  DRT  or  DAM  list  is  recorded  in  a  rough 
format  onto  U-matic  videotape.  The  rough  format  is  edited  into  a  final 
finished  format  with  continuous  SMPTE  time  code,  a  serial  number,  and 
a  consistent  speaker/list  announcement  format. 

It  is  highly  recommended  that  the  operation  manuals  be  read  carefully 
before  any  attempt  is  made  to  use  this  digital  audio  editing  system 
(Refs.  14,  15,  17,  18). 

8.3.1  Initial  Calibration 

A  IKhz  sine  wave  input  to  the  Ampex  is  adjusted  so  as  to  give  a  OVu 
reading  on  the  recorder's  track  meter  being  assigned  to  the  PCM- 100 
channel  left.  This  signal  passes  through  the  attenuator  which  is  set 
to  give  15  db  of  padding.  The  input  level  of  the  PCM  system  is 
adjusted  to  offset  that  15  db  and  give  a  reading  of  OVu  on  the  PCM- 100 
meter (emphasis  disengaged) .  The  line  level  at  the  Otari  is  also  set  to 
OVu.  In  this  state,  the  system  provides  up  to  15  db  of  boost  or  over 
85  db  attenuation  of  the  analog  master  signal  before  digitizing.  The 
procedure  is  repeated  for  the  analog  track  assigned  to  the  PCM- 100 
channel  right. 


8.3.2  Measuring 

Master  tapes  are  played  back  on  the  Ampex  with  their  reference  tones 
set  at  OVu.  The  signal  flows  through  the  system  and  is  measured  by  the 
EPL-S/N  program  implemented  on  the  PDP  11/34.  The  11/34  is  manually 
triggered  to  begin  sampling  directly  after  the  first  word  of  the  DRT 
list  (in  the  case  of  DAM  lists,  the  11/34  is  triggered  directly  after 


the  first  sentence  is  completed) .  The  emphasis  circuitry  of  the 
PCM-100  is  activated  before  mearuring  begins.  Sampling  continues  for 
forty  seconds  and  is  then  automatically  terminated. 

The  resultant  EPL  value  determines  the  amount  of  boost/cut  necessary 
to  reach  the  9db  EPL  target  figure.  The  Attenuator  system  is  adjusted 
accordingly  and  the  material  is  remeasured.  This  process  is  repeated 
until  the  9db  figure  is  reached.  The  speech  material  is  then  digitally 
recorded.  A  hardcopy  of  the  data  generated  from  the  analog  master 
presented  at  its  original  level  and  the  final  digital  version  are 
produced. 


8.3.3  Digital  Recording 

All  manipulations  of  the  digital  audio  mastering  system  take  place  at 
the  remote  DAE- 1100  Keyboard.  The  order  of  operations  for  recording  of 
audio  signals  presented  at  the  analog  inputs  of  the  PCM-100  are  as 
follows ; 

In  the  editor  section,  press  Recorder .  This  assigns  the  video  out  of 
the  PCM-100  to  the  U-matic  videocassette  deck  designated  as  Recorder. 
Insert  a  blank  U-Matic  cassette  into  the  recorder  VCR.  Be  sure  to  take 
up  any  slack  by  turning  the  reels  clockwise  manually.  Press  CUT  IN  REC 
and  FVD  on  the  keyboard.  The  Recorder  VCR  will  now  engage  and  the 
display  will  begin  incrementing  as  the  time  .  le  progresses.  It  is 
necessary  to  record  20-30  seconds  of  time  code  before  any  audio 
material.  The  analog  ch.  2  of  the  recorder  VCR  is  used  for  SMPTE  time 
code  and  its  Vu  meter  should  be  deflecting  to  OVu.  The  SMPTE  generator 
does  not  halt  when  the  recording  function  is  disengaged. 

WHENEVER  TAPE  IS  STOPPED  AND  RECORDING  RECOMMENCES, 

20-30  SECONDS  OF  SMPTE  TIME  CODE  MUST  BE  RECORDED 
BEFORE  ANY  DIGITAL  AUDIO  INFORMATION.  FAILURE  TO  DO 
THIS  WILL  RESULT  IN  AN  INABILITY  TO  EDIT. 

It  is  not  necessary  to  have  continuous  code  throughout  the  tape, 
however,  the  values  should  continually  ascend  and  there  can  be  no  more 
than  twelve  hours  difference  between  two  adjacent  sections. 

Along  with  audio  information  and  SMPTE  time  code,  there  exists  the 
possibility  to  record  a  series  of  number  values  called  user  bits. 
These  bits  allow  the  user  to  record  a  unique  number  value  between  0 
and  99999999  to  coincide  with  any  tape  or  portion  of  tape  desired. 
This  value  may  be  viewed  during  any  slow  transport  speed,  including 
shuttles.  Simply  press  the  User  Bits  button  and  the  Hold  button  in  the 
recorder  section  (you  must  have  a  tape  in  the  recorder  to  engage  this 
function) ,  the  right-most  digit  of  the  recorder  display  will  blink  and 
you  may  select  your  eight-number  pattern  from  the  row  of  numeral 
buttons  at  the  top  middle  of  the  keyboard.  Press  the  Hold  button  again 
when  the  pattern  is  complete.  These  user  bits  will  be  copied 
continuously  on  any  tapes  recorded  on  until  the  editing  system  is  shut 
down  or  the  user  bits  reset. 
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8.3.4  Editing 

The  Sony  digital  mastering  system  edits  in  a  tail-to-head  fashion. 
Aside  from  the  above  mentioned  note  on  SMPTE  time  code,  the  only 
prerequisite  is  a  continuous  length  of  time  code  on  the  destination 
tape  (Recorder) .  The  editing  system  has  a  preroll  function  that  uses 
the  SMPTE  time  code  to  rewind  to  5,  10,  or  30  seconds  before  the  edit 
point  (user  selectable) .  There  must  be  a  continuous  length  of  time 
code  to  meet  the  desired  preroll  length  on  both  source  and  destination 
tapes . 

During  editing  the  four  switches  of  the  PCM- 100  must  be  configured  as 
follows : 


EXT  SYNC :  INT 
REC  DATA:  EXT 
PB  DATA:  EXT 
REC  MASTER:  REC 

This  allows  the  processor  to  access  data  from  the  video  decks  rather 
than  from  the  analog  inputs . 

Insert  a  blank  cassette  into  the  VCR  assigned  as  recorder.  Record  the 
introductory  time  code.  Rewind  and  playback.  Select  an  edit  out  point 
by  pressing  the  Edit  Point  Out  button  in  the  editor  section  of  the 
keyboard.  Once  an  edit  button  is  pushed,  the  editor  dumps  3  seconds 
from  either  side  of  that  point  into  memory.  From  here,  the  6  seconds 
may  be  monitored  aurally  at  normal  or  half  speeds  and  visually  seen  as 
points  on  a  time  line.  With  the  manual  control  the  6  second  fragment 
can  be  manipulated  at  any  speed.  When  the  exact  spot  is  found  the  edit 
point  may  be  locked  by  pressing  Manual  and  Edit  Point  Out 
simultaneously . 

Load  the  rough  format  EPL  equalized  Digital  cassette  into  the  VCR 
deemed  as  player.  Press  the  Player  button  in  the  editing  section  of 
the  keyboard.  Start  the  player  forward  and  select  the  edit  in  point  by 
pressing  the  Edit  Point  In  button.  Once  again  the  editor  will  dump  3 
seconds  from  either  side  of  that  point  into  memory. .  Use  the  search 
modes  to  pinpoint  and  lock  in  by  pressing  Manual  and  Edit  In 
simultaneously . 

With  both  edit  points  memorized, press  Preview.  This  gives  a  dry  run  of 
the  edit.  If  the  splice  is  unsatisfactory.  The  Time  Offset  function 
may  be  used  to  move  either  point  up  to  +  or  -  30  seconds.  Any  number 
of  previews  and  time  offsets  may  be  made.  Once  a  desired  edit  is 
found,  press  Auto  Edit  and  Preview.  The  recording  will  be  generated 
the  same  as  the  last  preview.  Once  all  the  desired  material  has  been 
rerecorded,  pressing  Pause  in  the  player  section  will  halt  both  decks. 
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A  continuous  series  of  tail-to-head  editing  can  be  performed  in  this 
manner.  The  editor  automatically  prerolls  and  synchronizes  time  code 
to  insure  a  continuous  SMPTE  reference.  User  bits  do  not  transfer 
during  editing  nor  do  they  match  the  recorder  tape's  value.  User  bits 
must  be  reset  prior  to  recording.  Emphasis  information  is  transferred 
during  editing 

Once  an  edit  is  complete  a  review  can  be  made  by  simply  pressing  the 
Review  button.  This  automatically  shuttles  the  record  deck  back  to  the 
preroll  position  of  the  last  edit  and  plays  from  there  through  the 
edit  point  and  continues. 


8.3.5  Analog  Source  Tapes 

The  original  analog  DRT/DAM  master  tape  series  resides  on  1/2"  4- track 
tape  stored  on  ten  inch  reels.  Tape  speed  is  15  inches/sec.  Tapes  are 
recorded  and  played  back  with  NAB  equalization.  Most  of  the  masters 
have  track  assignments  that  consist  of  the  primary  microphone  on  two 
tracks,  an  Altec  659A  Dynamic  microphone  track  and  a  Grayson- Stadler 
Throat  Mic  track.  Most  of  the  analog  masters  have  the  following 
information  in  the  following  order: 

1)  1  KHz  reference  tone 

2)  Vocal  announcement  of  tape  title,  environment,  speaker 
number  and  gender,  and  primary  microphone. 

(primary  microphone  channel  only) 

3)  1  KHz  reference  tone 

4)  Speaker  1  DRT  list  announcement 

5)  Speaker  1  DRT  list 

6)  1  KHz  reference  tone 

7)  Speaker  2  DRT  list  announcement 

8)  Speaker  2  DRT  list 

9)  1  KHz  reference  tone 

10)  Speaker  3  DRT  list  announcement 

11)  Speaker  3  DRT  list 

12)  Speaker  1  DAM  list  announcement 

13)  Speaker  1  DAM  list 

14)  Speaker  2  DAM  list  announcement 

15)  Speaker  2  DAM  list 

16)  Speaker  3  DAM  list  announcement 

17)  Speaker  3  DAM  list 

DAM  lists  and  announcements  only  appear  on  the  tapes  of  the  first  DRT 
scramblings  for  that  particular  speaker  set/environment  combination. 
Only  the  first  reference  tone  on  the  masters  is  used  to  set  playback 
level  for  the  entire  tape.  Subsequent  reference  tones  vary  widely 
between  themselves  and  their  relationship  to  the  speech  material. 
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8.3.6  Reference  Tones 

It  is  imperative  that  record  levels  do  not  saturate  the  PCM  A/D. 
Unlike  analog  audio,  there  is  no  grey  area  where  a  portion  of  the 
signal  may  be  allowed  to  extend  past  the  dynamic  ceiling.  If  the  peak 
level  meters,  used  on  the  PCM- 100  processor,  indicate  overlevel  than 
that  material  is  distorted  beyond  the  use  of  this  facility.  With  this 
in  mind,  the  digital  master  series  has  reference  tones  that  allow 
speech  signals  to  be  recorded  at  a  universal  level,  making  full  use  of 
the  A/D  -  D/A  range  and  also  to  be  played  back  at  levels  acceptable 
for  processor  input  or  analog  dubbing. 

The  9dBm  EPL  figure  is  used  as  the  universal  digital  master 
presentation  level.  Speech  peaks  will  often  approach,  but  rarely 
exceed,  5dB  above  OVu  (OVu  -  +4dBm) .  This  allows  lOdB  of  headroom  on 
the  PCM- 100.  A  5KHz  reference  tone  is  recorded  on  the  masters  and 
reads  OVu  on  the  PCM- 100  (-ISdbVu  from  saturation).  See  Fig  8.2. 


dBVu 


9dBm  EPL 
Speech  Peak 
Limit 

SKHz  Tone 


Analog 
Speech  Peadc 
Limit 
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A  second  reference  is  a  IKHz  tone  that  reads  +5dBVu  on  the  PCM- 100 
(-lOdB  Vu  from  saturation).  This  tone  is  used  for  generating  analog 
dubs.  To  allow  speech  peaks  to  consistently  reach  levels  of  +5dB  above 
OVu  on  a  conventional  analog  recording  deck  invites  too  much  of  a 
chance  to  distort  important  speech  information.  Use  of  this  IKHz  tone 
as  a  reference  for  a  OVu  input  to  an  analog  deck  insures  speech  peaks 
approaching  but  rarely  passing  OVu. 

Reference  tones  are  recorded  using  an  oscilloscope  and  an  electronic 
counter  to  insure  quality  and  consistency  of  the  signal,  the  Microdot 
brand  sine  wave  generator  was  used  to  produce  two  reference  tones. 
These  sine  waves,  IKHz  at  lOdb  below  PCM  saturation  and  5KHz  at  OVu, 
were  recorded  digitally  in  lengthy  sections.  When  the  masters  are 
edited  into  their  final  formats,  sections  of  these  tones  are  digitally 
dubbed  in  providing  consistent  tones  throughout  the  master  series. 
These  new  tones  were  recorded  to  insure  full  quality,  stable 
references . 


8.3.7  Dubb ing 

Working  copies  residing  on  Beta  videocassette  will  be  copied  using  the 
PCM-Fl.  The  video  output  of  a  U-matic  deck  is  sent  to  the  video  input 
of  the  FI.  The  BETA  video  deck,  model  SL2000,  will  be  plugged  into  the 
"copy  out"  port  and  the  direct  copy  function  of  the  PCM-Fl  will  be 
used.  Begin  recording  on  the  Beta  deck  and  start  the  U-matic  deck.  The 
FI  will  produce  a  direct  digital  dub  of  the  U-matic  on  Beta  with  the 
exact  same  levels  and  frequency  response.  The  FI  will  also  provide  an 
analog  output  for  monitoring  (Fig.  8.3). 


Figure  8.3  Digital  Dubbing  Configuration 


Analog  dubs  are  produced  directly  from  the  analog  outputs  of  the 
PCM- 100.  The  5K  Hz  tone  is  not  dubbed  and  the  IK  Hz  reference  tone  is 
set  to  OVu  on  the  input  side  of  the  analog  deck.  This  insures  speech 
peaks  approaching,  but  rarely  surpassing  OVu  (-(-4dBm)  . 
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APPENDIX  A 


DRT/DAM  DIGITAL  MASTER  LIBRARY 


ENVIRONMENT 
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Rome  Air  Development  Center 


RADC  plans  and  executes  research,  development,  test  and 
selected  acquisition  programs  m  support  of  Command,  Control, 
Communications  and  Intell^ence  (C*I)  activities.  Technical  and 
engineering  support  within  areas  of  competence  is  provided  to 
ESD  Program  Offices  (POs)  and  other  ESD  elements  to 
perform  effective  acquisition  of  (PI  systems.  The  areas  of 
technical  competence  include  communications,  command  and 
control,  battle  management  information  processing,  surveillance 
sensors,  intelligence  data  collection  and  handling,  solid  state 
sciences,  electromagnetics,  and  propagation,  and  electronic 
reliability /maintainability  and  compatibility. 


